Search

Federated learning (FL) enables edge devices to cooperatively train models without exposing their raw data. However, implementing a practical FL system at the network edge mainly faces three challenges: label noise, data non-IIDness, and device heterogeneity, which seriously harm model performance and slow down convergence speed. Unfortunately, none of the existing works tackle all three challenges simultaneously. To this end, we develop a novel FL system, called Aorta, which features adaptive dataset construction and aggregation weight assignment. On each client, Aorta first calibrates potentially noisy labels and then constructs a training dataset with low noise, balanced distribution, and proper size. To fully utilize limited data on clients, we propose a global model guided method to select clean data and progressively correct noisy labels. To achieve balanced class distribution and proper dataset size, we propose a distribution-and-capability-aware data augmentation method to generate local training data. On the server, Aorta assigns aggregation weights based on the quality of local models to ensure that high-quality models have a greater influence on the global model. The model quality is measured through its cosine similarity with a benchmark model, which is trained on a clean and balanced dataset. We conduct extensive experiments on four datasets with various settings, including different noise types/ratios and non-IID types/levels. Compared to the baselines, Aorta improves model accuracy up to 9.8% on the datasets with moderate noise and non-IIDness, while providing a speedup of 4.2× on average when achieving the same target accuracy.

In a multi-tenant cloud, the cloud service provider (CSP) leases physical resources to tenants in the form of virtual machines (VMs) with an agreed service level agreement (SLA). As the most important indicator of SLA, we should guarantee the service availability of tenants when placing the VMs. However, previous works about VM placement mainly concentrate on optimizing the cloud resource utilization, but only a few works consider the service availability by measuring the hardware availability. In fact, abnormal tenants can make the corresponding service unavailable by launching network attacks. That is, both the hardware availability and the tenant uncertainty will affect the service availability of VMs on physical machines (PMs). Without considering this factor, the CSP may fail to meet the tenant’s SLA requirements, leading to a reduction in revenue. To solve such a problem, this paper considers the service availability in terms of both the hardware availability and the tenant uncertainty, and studies the service availability-guaranteed VM placement in multi-tenant clouds (SAG-VMP) problem. This problem is very challenging since the service availability actually changes with the tenants served on the PM. To address this issue, we propose a two-phase approach: PM assignment and VM placement. The first phase determines the availability of each PM through a long-term tenant-PM mapping algorithm and the second phase places each VM on a PM that meets the service availability requirement based on a primal-dual online algorithm. Two algorithms with bounded approximation factors are proposed for these two phases, respectively. Both small-scale experiment results and large-scale simulation results show the superior performance of our proposed algorithms compared with other alternatives.

With the increase in both the model size and dataset size of distributed training (DT) tasks, communication between the workers and parameter servers (PSs) in a cluster has become a bottleneck. In-network aggregation (INA) enabled by programmable switches has been proposed as a promising solution to alleviate the communication bottleneck. However, existing works focused on in-network aggregation implementation based on simple DT placement and fixed routing policies, which may lead to a large communication overhead and inefficient use of resources (e.g., storage, computing power and bandwidth). In this paper, we propose PARING, the first-of-its-kind INA approach that jointly optimizes DT task placement and routing in order to reduce traffic volume and minimize communication time. We formulate the problem as a nonlinear multi-objective mixed-integer programming problem, and prove its NP-Hardness. Based on the concept of Steiner trees, an algorithm with bounded approximation factors is proposed for this problem. Large-scale simulations show that our algorithm can reduce communication time by up to 81.0% and traffic volume by up to 19.1% compared to the state-of-the-art algorithms.

Data generated at the network edge can be processed locally by leveraging the emerging technology of Federated Learning (FL). However, non-IID local data will lead to degradation of model accuracy and the heterogeneity of edge nodes inevitably slows down model training efficiency. Moreover, to avoid the potential communication bottleneck in the parameter-server-based FL, we concentrate on the Decentralized Federated Learning (DFL) that performs distributed model training in Peer-to-Peer (P2P) manner. To address these challenges, we propose an asynchronous DFL system by incorporating neighbor selection and gradient push, termed AsyDFL. Specifically, we require each edge node to push gradients only to a subset of neighbors for resource efficiency. Herein, we first give a theoretical convergence analysis of AsyDFL under the complicated non-IID and heterogeneous scenario, and further design a priority-based algorithm to dynamically select neighbors for each edge node so as to achieve the trade-off between communication cost and model performance. We evaluate the performance of AsyDFL through extensive experiments on a physical platform with 30 NVIDIA Jetson edge devices. Evaluation results show that AsyDFL can reduce the communication cost by 57% and the completion time by about 35% for achieving the same test accuracy, and improve model accuracy by at least 6% under the non-IID scenario, compared to the baselines.

Distributed training includes two important operations: gradient transmission and gradient aggregation, which will consume massive bandwidth and computing resources. To achieve efficient distributed training, one must overcome two critical challenges: heterogeneity of bandwidth resources and limitation of computing resources among compute nodes. Existing architectures based on Parameter Server (PS) and All-Reduce (AR) fail to cope with these challenges because the PS will aggregate gradients from all workers and suffers from bandwidth bottlenecks, while AR intends to alleviate bandwidth bottlenecks at the PS, but the workers need to process many gradient packets thus can be overloaded. To address these shortcomings, we design a new distributed training system called ALEPH. In the control plane, ALEPH uses an efficient algorithm to group workers into clusters with different sizes so as to fully utilize heterogeneous bandwidth. We show that the proposed algorithm can achieve a good approximation performance. In the data plane, ALEPH leverages, for the first time, extended Berkeley Packet Filter (eBPF) programs to aggregate and forward gradient packets to reduce computation overhead. We show how to overcome several hurdles in using eBPF for distributed training. We implement ALEPH and evaluate its performance on a small-scale testbed and large-scale simulations. Experimental results show that ALEPH reduces training time by 20%-31% and increases bandwidth utilization by 88% compared with state-of-the-art frameworks.

The emerging Federated Learning (FL) permits all workers (e.g., mobile devices) to cooperatively train a model using their local data at the network edge. In order to avoid the possible bottleneck of conventional parameter server architecture, the decentralized federated learning (DFL) is developed on the peer-to-peer (P2P) communication. Non-IID issue is a key challenge in FL and will significantly degrade the model training performance. To this end, we propose a personalized solution called TOPFL, in which only parts of the local models (not the entire models) are shared and aggregated. Moreover, considering the limited communication bandwidth on workers, we propose a topology construction algorithm to accelerate the training process. To verify the convergence of the decentralized training framework, we theoretically analyze the impact of the data heterogeneity and topology on the convergence upper bound. Extensive simulation results show that TOPFL can achieve 2.2× speedup when reaching convergence and 5.8% higher test accuracy under the same resource consumption, compared with the baseline solutions.

The massive data from mobile and embedded devices have huge potential for training machine learning models. Decentralized machine learning (DML) can avoid the inherent bottleneck of the parameter server (PS) by collaboratively training models in a device-to-device (D2D) fashion. However, the previous DML works often assume that the local data are fully annotated with ground-truth labels, which is unrealistic for many Internet of Things (IoT) applications. This arises a new practical DML scenario, namely semi-supervised DML, where the local data of distributed workers are partially labeled in the D2D network. The existing semi-supervised learning techniques are proposed for standalone or the PS architecture, which ignore the impact of D2D topology on the performance of semi-supervised learning. Thus, they cannot adequately leverage the unlabeled data of decentralized workers, leading to performance degradation. Herein, we propose a novel framework, called SSD, to address the problem of semi-supervised DML by exploiting D2D cooperation. The key insight behind SSD is that neighbor selection has a crucial impact on pseudo-label quality and communication overhead. In SSD, each worker adaptively selects its neighbors with high-quality models and similar data distribution under communication resource constraints, which helps to generate high-confidence pseudo-labels for local unlabeled data and further boosts the DML performance. Extensive empirical evaluations on both testbed and simulated environments show that SSD significantly outperforms other baselines.

Federated learning (FL) is an effective approach to train models collaboratively among distributed edge nodes (i.e., workers) while facing three crucial challenges, edge heterogeneity, resource constraint, and Non-IID data. Under the parameter server (PS) architecture, a single parameter server may become the system bottleneck and cannot well deal with the edge heterogeneity, while the peer-to-peer (P2P) architecture causes significant communication consumption to achieve satisfactory training performance. To this end, hierarchical aggregation (HA) architecture is proposed to cluster workers to tackle the edge heterogeneity and reduce communication consumption for FL. However, the existing researches on HA architecture cannot provide a unified clustering approach for various inter-cluster aggregation patterns (e.g., centralized or decentralized structure, synchronous or asynchronous mode). In this paper, we explore the quantitative relationship between the convergence bounds of different inter-cluster patterns and several factors, e.g., data distribution, frequency of clusters participating in inter-cluster aggregation (for asynchronous modes), and inter-cluster topology (for decentralized structures). Based on the convergence bounds, we design a unified clustering algorithm FedUC to organize workers for different patterns. Experimental results on classical models and datasets show that FedUC can greatly accelerate the model training of different patterns by 1.79-7.39× compared with the state-of-the-art clustering methods.

Federated semi-supervised learning (FSSL) has been proposed to address the insufficient labeled data problem by training models with pseudo-labeling. In previous FSSL systems, a single global model is always trained without an equivalent generalization ability for the clients under the non-IID setting. Accordingly, model personalization methods have been proposed to overcome this problem. Intuitively, seeking labeling assistance from other clients with similar data distribution, i.e., model migration, can effectively improve the personalization on the clients with scarce labeled data. However, previous works require to migrate a pre-fixed number of models among the clients, causing unnecessary resource waste and accuracy degradation due to resource heterogeneity. Considering that the number of model migrations and the quality of pseudo-labels have a significant impact on the training performance (e.g., efficiency and accuracy), we propose a novel personalized FSSL system, called Ferrari, to boost the efficiency of pseudo-labeling and training accuracy through adaptive model migrations among the clients. Specifically, Ferrari first generates the similarity-based ranking using a Gaussian KD-Tree, considering the varied data distributions among the clients. Combined with the ranking and clients’ heterogeneous resource constraints, Ferrari then adaptively determines the proper model migration policy and confidence thresholds for high-quality pseudo-labeling and personalized training for clients. Extensive experiments on a physical platform show that Ferrari provides a 1.2$\sim 5.5\times$∼5.5× speedup without sacrificing model accuracy, compared to existing methods.

To approach the challenges of non-IID data and limited communication resource raised by the emerging federated learning (FL) in mobile edge computing (MEC), we propose an efficient framework, called FedMigr, which integrates a deep reinforcement learning (DRL) based model migration strategy into the pioneer FL algorithm FedAvg. According to the data distribution and resource budgets, our FedMigr will intelligently guide one client to forward its local model to another client after local updating, before directly sending the local models to the server for global aggregation as in FedAvg. Intuitively, migrating a local model from one client to another is equivalent to training the model over more data from different clients, alleviating the influence of non-IID issue. To this end, we propose an experience-driven method to make proper decisions for model migrations while satisfying the resource constraints. We also prove that FedMigr can help to reduce the parameter divergences between different local models and the global model from a theoretical perspective under the non-IID setting. Extensive experiments on three popular benchmark datasets demonstrate that FedMigr can achieve an average accuracy improvement of around 13%, and reduce bandwidth consumption for global communication by 42% on average, compared with the baselines.

The surging scale of distributed training (DT) incurs significant communication overhead in datacenters, while a promising solution is in-network aggregation (INA). It leverages programmable switches (e.g., Intel Tofino switches) for gradient aggregation to accelerate DT tasks. Due to switches’ limited on-chip memory size, existing solutions try to design the memory sharing mechanism for INA. This mechanism requires gradients to arrive at switches synchronously, while network dynamics make it common for the asynchronous arrival of gradients, resulting in existing solutions being inefficient (e.g., massive communication overhead). To address this issue, we propose GOAT, the first-of-its-kind work on gradient scheduling with collaborative in-network aggregation, so that switches can efficiently aggregate asynchronously arriving gradients. Specifically, GOAT first partitions the model into a set of sub-models, then decides which sub-model gradients each switch is responsible for aggregating exclusively and to which switch each worker should send its sub-model gradients. To this end, we design an efficient knapsack-based randomized rounding algorithm and formally analyze the approximation performance. We implement GOAT and evaluate its performance on a testbed consisting of 3 Intel Tofino switches and 9 servers. Experimental results show that GOAT can speed up the DT by $1.5 \times $ compared to the state-of-the-art solutions.

The emerging paradigm of federated learning (FL) strives to enable devices to cooperatively train models without exposing their raw data. In most cases, the data across devices are non-independently and identically distributed in FL. Thus, the local models trained over different data distributions will inevitably deviate from the global optima, which induces optimization inconsistency and even hurts global convergence. Moreover, the resource-constrained devices with heterogeneous training capacities (e.g., computing and communication) further slow down the convergence rate. To this end, we introduce an FL framework with adaptive data sampling and local training, namely FAST. Specifically, even without devices’ private data distributions, FAST enables each device to sample different rates of data points from each of its local classes to rebuild a dataset for training, thus adjusting the convergence direction of the aggregated global model to be closer to the global optima. The theoretical analysis shows that the convergence bound depends on the sampling rates as well as the number of local iterations executed on the sampled data. To achieve resource-effective and convergence-guaranteed FL, we then design an online learning algorithm that jointly optimizes the data sampling and local training strategies so as to encourage the decrease of global loss under the given time budget. Extensive experiments on physical and simulated environments show that, FAST improves the model accuracy by about 1.55%-6.78% given the same time budget, and accelerates training by about 1.39-5.89× with the same target accuracy, compared with the baselines.

Automatic crack detection in construction facilities is a challenging yet crucial task. However, existing deep learning (DL)-based semantic segmentation methods for this field are based on fully supervised learning models and pixel-level manual annotation, which are time-consuming and labor-intensive. To solve this problem, this paper proposes a novel crack semantic segmentation network using weakly supervised approach and mixed-label training strategy. Firstly, an image patch-level classifier of crack is trained to generate a coarse localization map for automatic pseudo-labeling of cracks combined with a thresholding-based method. Then, we integrated the pseudo-annotated with manual-annotated samples with a ratio of 4:1 to train the crack segmentation network with a mixed-label training strategy, in which the manual labels were assigned with a higher weight value. The experimental data on two public datasets demonstrate that our proposed method achieves a comparable accuracy with the fully supervised methods, reducing over 65% of the manual annotation workload.

The emerging Personalized Federated Learning (PFL) methods aim to produce personalized models for different users, so as to keep track of their individualized requirements in Edge Computing (EC). The centralized PFL methods may suffer from the communication bottleneck and single point of failure. As an alternative solution, the decentralized PFL (DPFL) methods are performed in a Peer-to-Peer (P2P) manner, and collaboratively train personalized models by model aggregation among congenial devices. However, these DPFL methods may incur high communication cost and low resource utilization induced by large-scale models. Herein, we take the communication constraint and heterogeneity into consideration and propose to realize communication-efficient DPFL with adaptive model pruning and neighbor selection. We theoretically analyze the convergence of the proposed DPFL method, and study the impacts of both model pruning and neighbor selection on training performance. Furthermore, we propose an efficient algorithm that combines model pruning and neighbor selection to achieve a trade-off between model quality and communication cost. Extensive simulation and testbed experiments on real-world datasets are conducted. The experimental results demonstrate that the proposed algorithm can improve the test accuracy by at most 13% and save the traffic consumption by 45.4% on average compared with the existing PFL methods.

Recently, edge AI has been launched to mine and discover valuable knowledge at network edge. Federated Learning, as an emerging technique for edge AI, has been widely deployed to collaboratively train models on many end devices in data-parallel fashion. To alleviate the computation/communication burden on the resource-constrained workers (e.g., end devices) and protect user privacy, Spilt Federated Learning (SFL), which integrates both data parallelism and model parallelism in Edge Computing (EC), is becoming a practical and popular approach for model training over distributed data. However, apart from the resource limitation, SFL still faces two other critical challenges in EC, i.e., system heterogeneity and context dynamics. To overcome these challenges, we present an efficient SFL method, named AdaSFL, which controls both local updating frequency and batch size to better accelerate model training. We theoretically analyze the model convergence rate and obtain a convergence upper bound regarding local updating frequency given a fixed batch size. Upon this, we develop a control algorithm to determine adaptive local updating frequency and diverse batch sizes for heterogeneous workers to enhance the training efficiency. The experimental results show that AdaSFL can reduce the completion time by about 43% and the network traffic consumption by about 31% for achieving the similar test accuracy, compared to the baselines.

In a commercial cloud, service providers (e.g., video streaming service provider) rent resources from cloud vendors (e.g., Google Cloud Platform) and provide services to cloud users, making a profit from the price gap. Cloud users acquire services by forwarding their requests to corresponding servers. In practice, as a common scenario, traffic dynamics will cause server overload or load-unbalancing. Existing works mainly deal with the problem by two methods: elastic resource provisioning and request updating. Elastic resource provisioning is a fast and agile solution but may cost too much since service providers need to buy extra resources from cloud vendors. Though request updating is a free solution, it will cause a significant delay, resulting in a bad users’ QoS. In this paper, we present a new scheme, called real-time request updating with elastic resource provisioning (TRUST), to help service providers pay less cost with users’ QoS guarantee in clouds. In addition, we propose an efficient algorithm for TRUST with a bounded approximation factor based on progressive-rounding. Both small-scale experiment results and large-scale simulation results show the superior performance of our proposed algorithm compared with state-of-the-art benchmarks.

There are two conventional methods to establish an entanglement connection in a Quantum Data Networks (QDN). One is to create single-hop entanglement links first and then connect them with quantum swapping, and the other is forwarding one of the entangled photons from one end to the other via all-optical switching at intermediate nodes to directly establish an entanglement connection. The two methods both have pros and cons. Respectively, the former method has a higher success probability of constructing entanglement link, but it would consume more quantum resources. The latter method, however, has a lower success probability to deliver a photon across multiple quantum links with fewer quantum resources. Accordingly, we are expecting to establish significantly more entanglement connections with limited quantum resources by first creating entanglement segments, each spanning multiple quantum link, using all-optical switching, and then connecting them with quantum swapping. In this paper, we design SEE, a Segmented Entanglement Establishment approach that seamlessly integrates quantum swapping and all-optical switching to maximize quantum network throughput. SEE first creates entanglement segments over one or multiple quantum links with all-optical switching, and then connect them with quantum swapping. Accordingly, SEE can theoretically outperform conventional entanglement link-based approaches. Large scale simulations show that SEE can achieve up to 100.00% larger throughput compared with the state-of-the-art entanglement link-based approaches, e.g., Redundant Entanglement Provisioning and Selection (REPS).

Federated Learning (FL) is a distributed model training framework that allows multiple clients to collaborate on training a global model without disclosing their local data in edge computing (EC) environments. However, FL usually faces statistical heterogeneity (e.g., non-IID data) and system heterogeneity (e.g., computing and communication capabilities), resulting in poor model training performance. To deal with the above two challenges, we propose an efficient FL framework, named FedBR, which integrates the idea of block-wise regularization and knowledge distillation (KD) into the pioneering FL algorithm FedAvg, for resource-constrained edge computing. Specifically, we first divide the model into multiple blocks according to the layer order of deep neural network (DNN). The server only sends some consecutive model blocks instead of an entire model to clients for communication efficiency. Then, the clients make use of knowledge distillation to absorb the knowledge of global model blocks to alleviate statistical heterogeneity during local training. We provide a theoretical convergence guarantee for FedBR and show that the convergence bound will decrease as the increasing number of model blocks sent by the server. Besides, since the increasing number of model blocks brings more computing and communication costs, we design a heuristic algorithm (GMBS) to determine the appropriate number of model blocks for clients according to their varied data distributions, computing, and communication capabilities. Extensive experimental results show that FedBR can reduce the bandwidth consumption by about 31%, and achieve an average accuracy improvement of around 5.6% compared with the baselines under heterogeneous settings.

The components and structures of the solid-electrolyte interphase (SEI) are critical for stable cycling of lithium metal batteries (LMBs). LiF has been widely studied as the dominant component of SEI, but Li2O, which has a much lower diffusion barrier for Li+, has rarely been investigated as the dominant component of SEI. The effect of Li2O-dominated SEI on electrochemical performance still remains elusive. Herein, an ultrastrong coordinated cosolvation diluent, 2,3-difluoroethoxybenzene (DFEB), is designed to modulate solvation structure and tailor Li2O-dominated SEI for stable LMBs. In the DFEB-based LHCE (DFEB-LHCE), DFEB intensively participates in the first solvation shell and synergizes with FSI–to tailor an Li2O-dominated inorganic-rich SEI which is different from the LiF-dominated SEI formed in conventional LHCE. Benefiting from this special SEI architecture, a high Coulombicefficiency (CE) of 99.58% in Li||Cu half cells, stable voltage profiles, and dense and uniform lithium deposition, as well as effective inhibition of Li dendrite formation in the symmetrical cell, are achieved. More importantly, the DFEB-LHCE can be matched with various cathodes such as LFP, NCM811, and S cathodes, and the Li||LFP full cell using DFEB-LHCE possesses 85% capacity retention after 650 stable cycles with 99.9% CE. Especially the 1.5 Ah practical lithium metal pouch cell achieves an excellent capacity retention of 89% after 250 cycles with a superb average CE of 99.93%. This work unravels the superiority of the Li2O-dominated SEI and the feasibility of tailoring SEI components through modulation of solvation structures.

The task of underwater image enhancement aims to improve the quality of the images and promote the visualization effect. Current methods still face some challenges, which include the difficulty to model the real ocean environment and the lack of simultaneous exploitation and modeling of both global and local information in images. Particularly, in resource-constrained underwater scenarios, models need to enhance images on embedded devices in real-time, which usually require them to strike a balance between the inference speed and accuracy. In this article, we put forward a real-time underwater image enhancement network (RUE-Net). Compared to previous advanced models using single branch structure or traditional U-shaped networks, it consists of parallel receptive field enhanced (RFE) module and fine-grained detail (FGD) module, which perform parallel modeling and fusion of image information at global and local scales. Furthermore, dense connections are established in each stage throughout the whole model. Compared with current models, RUE-Net shows the best performance in the image enhancement task in LSUI, UFO-120, and EUVP datasets. Furthermore, based on the RUE-Net, we introduce a simultaneous enhancement and super-resolution (SESR) multitask model. The experiments prove RUE-Net has an excellent capability for image restoration and super-resolution reconstruction in UFO-120 and USR-248 datasets. RUE-Net is also deployed on the Jetson Orin Nano, which achieves an impressive inference speed of up to 51 frames/s(FPS), demonstrating its value in underwater robot with limited computing capabilities. The code is available at https://github.com/GuocunWang/RUE-Net.

Cloud infrastructure has gradually displayed a tendency of geographical distribution in order to provide anywhere, anytime connectivity to tenants all over the world. The tenant task placement in geo-distributed clouds comes with three critical and coupled factors: regional diversity in electricity prices, access delay for tenants, and traffic demand among tasks. However, existing works disregard either the regional difference in electricity prices or the tenant requirements in geo-distributed clouds, resulting in increased operating costs or low user QoS. To bridge the gap, we design a cost optimization framework for tenant task placement in geo-distributed clouds, called TanGo. However, it is non-trivial to achieve an optimization framework while meeting all the tenant requirements. To this end, we first formulate the electricity cost minimization for task placement problem as a constrained mixed-integer non-linear programming problem. We then propose a near-optimal algorithm with a tight approximation ratio $(1-1/e)$ using an effective submodular-based method. Results of in-depth simulations based on real-world datasets show the effectiveness of our algorithm as well as the overall 10%-30% reduction in electricity expenses compared to commonly-adopted alternatives.

Traditional Federated Learning (FL) is a promising paradigm that enables massive edge clients to collaboratively train deep neural network (DNN) models without exposing raw data to the parameter server (PS). To avoid the bottleneck on the PS, Decentralized Federated Learning (DFL), which utilizes peer-to-peer (P2P) communication without maintaining a global model, has been proposed. Nevertheless, DFL still faces two critical challenges, i.e., limited communication bandwidth and not independent and identically distributed (non-IID) local data, thus hindering efficient model training. Existing works commonly assume full model aggregation at periodic intervals, i.e., clients periodically collect models from peers. To reduce the communication cost, these methods allow clients to collect model(s) from selected peers, but often result in a significant degradation of model accuracy when dealing with non-IID data. Alternatively, the layer-wise aggregation mechanism has been proposed to alleviate communication overhead under the PS architecture, but its potential in DFL remains rarely explored yet. To this end, we propose an efficient DFL framework YOGA that adaptively performs layer-wise model aggregation and training. Specifically, YOGA first generates the ranking of layers in the model according to the learning speed and layer-wise divergence. Combining with the layer ranking and peers’ status information (i.e., data distribution and communication capability), we propose the max-match (MM) algorithm to generate the proper layer-wise model aggregation policy for the clients. Extensive experiments on DNN models and datasets show that YOGA saves communication cost by about 45% without sacrificing the model performance compared with the baselines, and provides 1.53- $3.5\times $ speedup on the physical platform.

At the network edge, federated learning (FL) has gained attention as a promising approach for training deep learning (DL) models collaboratively across a large number of devices while preserving user privacy. However, FL still faces specific challenges related to the limited, heterogeneous and dynamic resources of devices. In most FL systems, all devices train the same model, while the devices with constrained resources, referred to as stragglers, will significantly slow down overall training process. It is intuitive to alleviate computation and communication load on the stragglers by training and transmitting a part of the model. Inspired by multi-exit models, we divide an original DL model into several non-overlapping blocks, which can be trained separately on the low-capability devices. Furthermore, we propose BOSE, a novel FL system that performs adaptive block-wise model training under resource constraints. Considering the diverse impacts of different blocks on model convergence and the varying training loads they incur, a naive block assignment strategy, e.g., uniformly random assignment, may not yield optimal model performance and fail to fully utilize available resources. To this end, we introduce two metrics, including learning speed and device-wise divergence, to measure the potential of blocks in promoting model convergence. Given resource budget, BOSE initially identifies a set of candidate blocks for each device and subsequently selects specific training blocks based on their potential for promoting model convergence. In general, blocks with higher potential are more likely to be chosen for training. Extensive experiments on a physical platform show that BOSE provides a $1.4\times \sim 3.8\times $ speedup without sacrificing model accuracy, compared to the baselines.

Federated learning (FL) has recently gained tremendous attention in edge computing and Internet of Things, due to its capability of enabling distributed clients to cooperatively train models while keeping raw data locally. However, the existing works usually suffer from limited communication resources, dynamic network conditions and heterogeneous client properties, which hinder efficient FL. To simultaneously tackle the above challenges, we propose a heterogeneity-aware FL framework, called FedCG, with adaptive client selection and gradient compression. Specifically, FedCG introduces diversity to client selection and aims to select a representative client subset considering statistical heterogeneity. These selected clients are assigned different compression ratios based on heterogeneous and time-varying capabilities. After local training, they upload sparse model updates matching their capabilities for global aggregation, which can effectively reduce the communication cost and mitigate the straggler effect. More importantly, instead of naively combining client selection and gradient compression, we highlight that their decisions are tightly coupled and indicate the necessity of joint optimization. We theoretically analyze the impact of both client selection and gradient compression on convergence performance. Guided by the convergence rate, we develop an iteration-based algorithm to jointly optimize client selection and compression ratio decision using submodular maximization and linear programming. On this basis, we propose the quantized extension of FedCG, termed Q-FedCG, which further adjusts quantization levels based on gradient innovation. Extensive experiments on both real-world prototypes and simulations show that FedCG and its extension can provide up to 6.4× speedup.

Federated learning (FL) has been widely adopted to train machine learning models over massive data in edge computing. Most works of FL employ pre-defined model architectures on all participating clients for model training. However, these pre-defined architectures may not be the optimal choice for the FL setting since manually designing a high-performance neural architecture is complicated and burdensome with intense human expertise and effort, which easily makes the model training fall into the local suboptimal solution. To this end, Neural Architecture Search (NAS) has been applied to FL to address this critical issue. Unfortunately, the search space of existing federated NAS approaches is extraordinarily large, resulting in unacceptable completion time on the resource-constrained edge clients, especially under the non-independent and identically distributed (non-IID) setting. In order to remedy this, we propose a novel framework, called Finch, which adopts hierarchical neural architecture search to enhance federated learning. In Finch, we first divide the clients into several clusters according to the data distribution. Then, some subnets are sampled from a pre-trained supernet and allocated to the specific client clusters for searching the optimal model architecture in parallel, so as to significantly accelerate the process of model searching and training. The extensive experimental results demonstrate the high effectiveness of our proposed framework. Specifically, Finch can reduce the completion time by about 30.6%, and achieve an average accuracy improvement of around 9.8% compared with the baselines.

Federated Learning (FL) has been widely adopted to process the enormous data in the application scenarios like Edge Computing (EC). However, the commonly-used synchronous mechanism in FL may incur unacceptable waiting time for heterogeneous devices, leading to a great strain on the devices’ constrained resources. In addition, the alternative asynchronous FL is known to suffer from the model staleness, which will lead to performance degradation of the trained model, especially on non-i.i.d. data. In this paper, we design a novel asynchronous FL mechanism, named FedLC, to handle the non-i.i.d. issue in EC by enabling the local collaboration among edge devices. Specifically, apart from uploading the local model directly to the server, each device will transmit its gradient to the other devices with different data distributions for local collaboration, which can improve the model generality. We theoretically analyze the convergence rate of FedLC and obtain the quantitative relationship between convergence bound and local collaboration. We design an efficient algorithm utilizing demand-list to determine the set of devices receiving gradients from each device. To handle the model staleness, we further assign different learning rates for various devices according to their participation frequency. The extensive experimental results demonstrate the effectiveness of our proposed mechanism.

Data generated at the network edge can be processed locally by leveraging the paradigm of edge computing (EC). Aided by EC, decentralized federated learning (DFL), which overcomes the single-point-of-failure problem in the parameter server based federated learning, is becoming a practical and popular approach for machine learning over distributed data. However, DFL faces two critical challenges, i.e., system heterogeneity and statistical heterogeneity introduced by edge devices. To ensure fast convergence with the existence of slow edge devices, we present an efficient DFL method, termed FedHP, which integrates adaptive control of both local updating frequency and network topology to better support the heterogeneous participants. We establish a theoretical relationship between local updating frequency and network topology regarding model training performance and obtain a convergence upper bound. Upon the convergence bound, we propose an optimization algorithm that adaptively determines local updating frequencies and constructs the network topology, so as to speed up convergence and improve the model accuracy. We evaluate the performance of FedHP through extensive simulation and testbed experiments. Evaluation results show that the proposed FedHP can reduce the completion time by about 51% and improve model accuracy by at least 5% in heterogeneous scenarios, compared with the baselines.

Quantum Data Networks (QDNs) are vital to building Distributed Quantum Computing (DQC) systems. Though several communication protocols have been proposed for QDNs, most of them are at the network layer or below. The only transport layer protocol used batch processing of requests for End-to-End (E2E) quantum data transmission. It not only limits the quantum resource utilization, more importantly, it cannot guarantee reliable E2E quantum data transmission. In this paper, we propose the first asynchronous transportation layer protocol, called AQTP, for QDNs to achieve high-speed and reliable E2E quantum data transmission. AQTP has several distinct features: 1) each quantum node locally allocates quantum resources in order to improve scalability; 2) requests are processed in an asynchronous manner, which results in a higher quantum resource utilization; and 3) it ensures reliable data transmission even if the teleportation operations fail. Extensive simulations show that compared with a batch processed transport layer protocol, AQTP can increase the network throughput by up to 82.97%, and reduce the Average Task Completion Time (ATCT) of DQC tasks by up to 94.69%.

In vehicular edge computing (VEC), the deployment of road side units (RSUs) along roads enables vehicles to offload computation-intensive tasks for efficient data processing. However, VEC poses unique challenges, including resource constraints on vehicles and RSUs, high vehicle mobility, and the large-scale nature of the infrastructure. Existing solutions, whether centralized or distributed, often suffer from longer decision-making times or task response times, making them unsuitable for vehicular scenarios. To address these challenges, this paper proposes a Fully Distributed Task Offloading (FDTO) decision-making scheme, which enables vehicles to iteratively adjust their offloading decisions based on resource utilization information obtained from neighboring RSUs. FDTO employs two different algorithms for decision adjustments: a greedy-based algorithm and a convex optimization-based algorithm. Theoretical analysis proves the convergence of the proposed algorithms to a global optimum through iterations. To evaluate the performance of FDTO, extensive simulations are conducted and the results demonstrate that the proposed algorithms offer near-optimal performance with a short decision-making time, reducing the average task response time by 50%-65% compared to existing algorithms.

With the growth of model/dataset/system size for distributed model training in datacenters, the widely used Parameter Server (PS) architecture suffers from communication bottleneck of gradient transmission. Recent works attempt to utilize programmable switches to implement in-network gradient aggregation and alleviate communication bottlenecks on PSs. Due to the limited on-chip memory of programmable switches, gradient transmission requires strict synchronization to achieve ideal aggregation performance. However, the distributed training system is usually heterogeneous in datacenters (e.g., computation and bandwidth heterogeneity), and the gradient will reach the aggregation nodes asynchronously, thereby seriously affecting the aggregation performance. To solve the above issue, we propose XAgg, which accelerates heterogeneous gradient aggregation by deploying the eXpress Data Path (XDP) based aggregator on servers. Specifically, the abundant idle memory on servers can cache the entire gradient, so as to effectively deal with asynchronous gradient transmission in heterogeneous scenarios. Moreover, XDP can provide high-performance and low-latency gradient aggregation. We conduct microbenchmark and testbed with real-world DNN models and datasets. Experimental results show that XAgg improves the gradient aggregation throughput by 3.3 $\times $ compared with TCP-based aggregation, reaching 100 Gbps with 10 CPU cores. In addition, XAgg reduces communication time by 49%-82% compared with state-of-the-art solutions.

Side-scan sonar (SSS) has become an important tool for ocean exploration due to its practicality and reliability. Existing approaches for SSS image classification mainly rely on deep convolutional neural networks (DCNNs). Although effective, DCNN-based methods fail to classify SSS images with specific target samples that are completely absent from the training data. In this work, a global context external-attention network (GCEANet) is proposed to produce pseudo-SSS images corresponding to the absent categories for zero-shot SSS image classification. First, the optical and acoustic images are taken as content and style image inputs to the network, respectively. Second, a global context (GC) block is proposed to extract the context semantics of the style features and fuse them with the content features. Finally, an external attention (EA) block is proposed to extract the potential correlation between content and style features to generate pseudo-SSS images with optical content. In addition, a contrastive loss based on image reconstruction is proposed to optimize the cross-modal feature space fusion. Extensive experiments demonstrate the effectiveness of our approach in efficiently generating realistic pseudo-SSS samples. The experimental results on real SSS sample datasets show that our method outperforms other state-of-the-art (SOTA) methods in the field of zero-shot SSS image classification. The code is available at https://github.com/baizhongyu/GCEANet.

The emerging Federated Learning (FL) permits all workers (e.g., mobile devices) to cooperatively train a model using their local data at the network edge. In order to avoid the possible bottleneck of conventional parameter server architecture, the decentralized federated learning (DFL) is developed on the peer-to-peer (P2P) communication. In DFL, model exchanging among workers is usually regarded as an atomic operation, which largely affects the total bandwidth consumption during model training. Given the limited communication resource on workers, model exchanging will pose a great challenge when meeting with the large-scale models. Herein, we propose to let workers exchange the intermediate results, instead of the entire model, with each other. We provide theoretical analysis of DFL based on intermediate result exchanging, which reveals the relationship between the training performance and the exchanging interval (i.e., the number of local updating iterations) of intermediate results. According to the convergence bound, we propose an adaptive exchanging interval (or frequency) algorithm called Fed-IR, which optimizes the trade-off between communication cost and training performance. Extensive simulation results show that compared with the model exchanging methods, our proposed algorithms can save communication traffic of around 42%$\sim$∼81% while still achieving the similar accuracy.

Federated learning (FL) has been widely applied to collaboratively train deep learning (DL) models on massive end devices (i.e., clients). Due to the limited storage capacity and high labeling cost, the data on each client may be insufficient for model training. Conversely, in cloud datacenters, there exist large-scale unlabeled data, which are easy to collect from public access (e.g., social media). Herein, we propose the Ada-FedSemi system, which leverages both on-device labeled data and in-cloud unlabeled data to boost the performance of DL models. In each round, local models are aggregated to produce pseudo-labels for the unlabeled data, which are utilized to enhance the global model. Considering that the number of participating clients and the quality of pseudo-labels will have a significant impact on the training performance, we introduce a multi-armed bandit (MAB) based online algorithm to adaptively determine the participating fraction and confidence threshold. Besides, to alleviate the impact of stragglers, we assign local models of different depths for heterogeneous clients. Extensive experiments on benchmark models and datasets show that given the same resource budget, the model trained by Ada-FedSemi achieves 3%$\sim$∼14.8% higher test accuracy than that of the baseline methods. When achieving the same test accuracy, Ada-FedSemi saves up to 48% training cost, compared with the baselines. Under the scenario with heterogeneous clients, the proposed HeteroAda-FedSemi can further speed up the training process by $1.3\times \sim 1.5\times$1.3×∼1.5×.

Federated learning (FL) has emerged as a promising distributed learning paradigm that enables a large number of mobile devices to cooperatively train a model without sharing their raw data. The iterative training process of FL incurs considerable computation and communication overhead. The workers participating in FL are usually heterogeneous and the workers with poor capabilities may become the bottleneck of model training. To address the challenges of resource overhead and system heterogeneity, this article proposes an efficient FL framework, called FedMP, that improves both computation and communication efficiency over heterogeneous workers through adaptive model pruning. We theoretically analyze the impact of pruning ratio on training performance, and employ a Multi-Armed Bandit based online learning algorithm to adaptively determine different pruning ratios for heterogeneous workers, even without any prior knowledge of their capabilities. As a result, each worker in FedMP can train and transmit the sub-model that fits its own capabilities, accelerating the training process without hurting model accuracy. To prevent the diverse structures of pruned models from affecting the training convergence, we further present a new parameter synchronization scheme, called Residual Recovery Synchronous Parallel (R2SP). Besides, our proposed framework can be extended to the peer-to-peer (P2P) setting. Extensive experiments on physical devices demonstrate that FedMP is effective for different heterogeneous scenarios and data distributions, and can provide up to 4.1× speedup compared to the existing FL methods.

With the broad deployment of distributed applications on clouds, east-west traffic is now dominating the majority of cloud networks. The existing communication solutions are tightly coupled with either the control plane (e.g., preprogrammed model) or the location of compute nodes (e.g., conventional gateway model). As a result, it is difficult to flexibly respond to the rapidly expanding networks and frequent abnormal events (e.g., burst traffic and device failures). Accordingly, they may not provide high-performance east-west forwarding while ensuring scalability and robustness. To address this issue, we design Zeta, a scalable and robust east-west forwarding framework with gateway clusters for hyperscale clouds. Zeta abstracts the traffic forwarding capability as a Gateway Cluster Layer, decoupled from the logic of control plane and the location of compute nodes. Specifically, Zeta adopts gateway clusters to support large-scale networks and cope with burst traffic. Moreover, a transparent Multi IPs Migration is proposed for fast recovery from unpredictable failures. We implement Zeta based on eXpress Data Path (XDP) and evaluate its scalability and robustness through comprehensive experiments with up to 100k container instances. Our evaluation shows that Zeta reduces the 99% RTT by $5.1 {\times }$ in burst video traffic, and reduces the gateway pure recovery delay by $10.8 {\times }$ compared with the state-of-the-art solutions.

As the scale of distributed training increases, it brings huge communication overhead in clusters. Some works try to reduce the communication cost through gradient compression or communication scheduling. However, these methods either downgrade the training accuracy or do not reduce the total transmission amount. One promising approach, called in-network aggregation, is proposed to mitigate the bandwidth bottleneck in clusters by aggregating gradients in programmable hardware (e.g., Intel Tofino switches). However, existing solutions mainly implement in-network aggregation through fixed (or default) routing paths, resulting in load imbalancing and long communication time. To deal with this issue, we propose GRID, the first-of-its-kind work on Gradient Routing with In-network Aggregation for Distributed Training. In the control plane, we present an efficient gradient routing algorithm based on randomized rounding and formally analyze the approximation performance. In the data plane, we realize in-network aggregation by carefully designing the logic of workers and programmable switches. We implement GRID and evaluate its performance on a small-scale testbed consisting of 3 Intel Tofino switches and 9 commodity servers. With a combination of testbed experiments and large-scale simulations, we show that GRID can reduce the communication time by 38.4%–60.1% and speed up distributed training by 17.4%–52.7% compared with state-of-the-art solutions.

Recently, mobile and embedded devices worldwide generate a massive amount of data at the network edge. To efficiently exploit the data from distributed devices, we concentrate on decentralized machine learning (DML), where the workers collaboratively train models under the peer-to-peer (P2P) setting. DML avoids the bottleneck of the parameter server (PS) by enabling the workers to exchange local models with their neighbors rather than the PS. However, DML still faces some key challenges, i.e., resource limitation, system heterogeneity, network dynamics and non-IID data. In this article, we design and implement MOTOR, an efficient DML mechanism that simultaneously addresses these challenges by applying model pruning and topology construction, thus accelerating DML. Specifically, MOTOR assigns different pruning ratios to heterogeneous workers. After model pruning, each worker will train and transmit a sub-model that fits its capabilities, reducing both computation and communication overhead. Besides, MOTOR dynamically constructs the network topology considering the time-varying network conditions and non-IID data distributions. We theoretically analyze the impact of pruning ratio and network topology on model training performance. Guided by the theoretical analysis, we develop a joint optimization algorithm for pruning ratio decision and topology construction to achieve the trade-off between resource overhead and training performance. We implement MOTOR on commercial devices and evaluate the performance with different DML tasks. Extensive experiments show that MOTOR achieves up to 4.2× speedup compared to the existing DML approaches.

Data generated at the network edge can be processed locally by leveraging the paradigm of Edge Computing (EC). Aided by EC, Federated Learning (FL) has been becoming a practical and popular approach for distributed machine learning over locally distributed data. However, FL faces three critical challenges, i.e., resource constraint, system heterogeneity and context dynamics in EC. To address these challenges, we present a training-efficient FL method, termed FedLamp, by optimizing both the Local updating frequency and model compression ratio in the resource-constrained EC systems. We theoretically analyze the model convergence rate and obtain a convergence upper bound related to the local updating frequency and model compression ratio. Upon the convergence bound, we propose a control algorithm, that adaptively determines diverse and appropriate local updating frequencies and model compression ratios for different edge nodes, so as to reduce the waiting time and enhance the training efficiency. We evaluate the performance of FedLamp through extensive simulation and testbed experiments. Evaluation results show that FedLamp can reduce the traffic consumption by 63% and the completion time by about 52% for achieving the similar test accuracy, compared to the baselines.

In edge computing (EC), federated learning (FL) enables massive devices to collaboratively train AI models without exposing local data. In order to avoid the possible bottleneck of the parameter server (PS) architecture, we concentrate on the decentralized federated learning (DFL), which adopts peer-to-peer (P2P) communication without maintaining a global model. However, due to the intrinsic features of EC, e.g., resource limitation and heterogeneity, network dynamics and non-IID data, DFL with a fixed P2P topology and/or an identical model compression ratio for all workers results in a slow convergence rate. In this paper, we propose an efficient algorithm (termed CoCo) to accelerate DFL by integrating optimization of topology Construction and model Compression. Concretely, we adaptively construct P2P topology and determine specific compression ratios for each worker to conquer the system dynamics and heterogeneity under bandwidth constraints. To reflect how the non-IID data influence the consistency of local models in DFL, we introduce the consensus distance, i.e., the discrepancy between local models, as the quantitative metric to guide the fine-grained operations of the joint optimization. Extensive simulations and testbed experiments show that CoCo achieves 10× speedup, and reduces the communication cost by about $50\%$50% on average, compared with the existing DFL baselines.

Solid-state lithium batteries with solid polymer electrolytes have recently attracted extensive attention due to their promising potential in high energy density and safety. However, the principal issues plaguing the solid polymer electrolytes are their restricted ionic conductivities at ambient temperature and the limited tolerance to the widely used high-voltage cathodes (such as LiNi0.8Mn0.1Co0.1O2, NCM811), thus limiting their practical applications seriously. In this regard, a superior polymer-in-salt solid electrolyte from poly(caprolactone)-graft-polyrotaxane (PGPE) is developed for high-voltage lithium batteries operated at room temperature. The PGPE displays remarkable electrochemical properties at room temperature, with an exceptional ionic conductivity of 4.89 × 10–4S cm–1and a lithium-ion transference number of approximately 0.64, stemming from the rapid segmental motions of PCL sidechains by the enhanced dynamics of the cyclic molecules along the axial polymer chain of polyrotaxane. More importantly, the PGPE demonstrates a high electrochemical oxidation voltage of ∼4.7 V, suggesting the excellent electrochemical stability of PGPE against the NCM811-based cathode. Owing to the dense LiF-rich CEI self-generated on the NCM811 particles in the cathode, the transition metal ion diffusion is successfully constrained and the PGPE is well protected from continuous decomposition. The PGPE also shows superior interfacial stability between the metallic Li and the electrolyte. As a result, the all-solid-state NCM811|PGPE|Li cell exhibits superior discharge capacity (196 mAh g–1) and extraordinary long-term cycling stability (74% capacity retention at 150 cycles) at 30 °C.

Recently, multi-material designs consisting of metal and FRP components are being increasingly used in vehicle body structures to reduce weight and thus energy consumption and emission. Because the most commonly used resistance spot welding (RSW) technology for body assembly cannot be applied to join sheet metals and FRPs, the usage of FRPs is strongly limited. Therefore, a new resistance insert spot welding (RISW) method was developed. During the manufacturing process of FRPs, here compression molding of PA6 GF40 materials, specially designed steel inserts are directly integrated into the FRP part. Later, via the steel inserts, the FRP part can be spot welded to other steel parts in the structure. In this work, the design and development process of RISW is presented. Starting with the design concept of inserts, welding tests were conducted, and the parameters for the welding simulation were calibrated. After that, the detailed insert design was performed using welding simulation and tests with several restrictions, such as insert dimensions, the temperature in FRPs, the weldability range, and joining strength. In the end, an appropriate insert geometry was found, which met all requirements. The mechanical properties of RISW joining are better than or equal to those of SPRs. The new RISW technology was also demonstrated on a real vehicle component. A seat cross member was designed using FRP with 42% weight reduction and produced by compression molding with 30 simultaneously integrated inserts. The cross member was finally successfully welded to a steel closing plate using RISW.