Search

Distributed training of deep neural networks faces three critical challenges: privacy preservation, communication efficiency, and robustness to fault and adversarial behaviors. Although significant research efforts have been devoted to addressing these challenges independently, their synthesis remains less explored. In this paper, we propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the emerging f-differential privacy (DP) and the Byzantine resilience of the proposed algorithm. Particularly, in terms of privacy guarantees, compared to the existing sign-based approach StoSign, the proposed method improves the dimension dependence on the gradient size and enjoys privacy amplification by mini-batch sampling while ensuring a comparable convergence rate. We also prove that TernaryVote is robust when less than 50% of workers are blind attackers, which matches that of SIGNSGD with majority vote. Extensive experimental results validate the effectiveness of the proposed algorithm.

Detecting unusual patterns in graph data is a crucial task in data mining. However, existing methods face challenges in consistently achieving satisfactory performance and often lack interpretability, which hinders our understanding of anomaly detection decisions. In this paper, we propose a novel approach to graph anomaly detection that leverages the power of interpretability to enhance performance. Specifically, our method extracts an attention map derived from gradients of graph neural networks, which serves as a basis for scoring anomalies. Notably, our approach is flexible and can be used in various anomaly detection settings. In addition, we conduct theoretical analysis using synthetic data to validate our method and gain insights into its decision-making process. To demonstrate the effectiveness of our method, we extensively evaluate our approach against state-of-the-art graph anomaly detection techniques on real-world graph classification and wireless network datasets. The results consistently demonstrate the superior performance of our method compared to the baselines.

Communication overhead has become one of the major bottlenecks in the distributed training of deep neural networks. To alleviate the concern, various gradient compression methods have been proposed, and sign-based algorithms are of surging interest. However, SIGNSGD fails to converge in the presence of data heterogeneity, which is commonly observed in the emerging federated learning (FL) paradigm. Error feedback has been proposed to address the non-convergence issue. Nonetheless, it requires the workers to locally keep track of the compression errors, which renders it not suitable for FL since the workers may not participate in the training throughout the learning process. In this paper, we propose a magnitude-driven sparsification scheme, which addresses the non-convergence issue of SIGNSGD while further improving communication efficiency. Moreover, the local update scheme is further incorporated to improve the learning performance, and the convergence of the proposed method is established. The effectiveness of the proposed scheme is validated through experiments on Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets.

The recent rapid growth in mobile data traffic entails a pressing demand for improving the throughput of the underlying wireless communication networks. Network node deployment has been considered as an effective approach for throughput enhancement which, however, often leads to highly non-trivial non-convex optimizations. Although convex approximation based solutions are considered in the literature, their approximation to the actual throughput may be loose and sometimes lead to unsatisfactory performance. With this consideration, in this paper, we propose a novel graph neural network (GNN) method for the network node deployment problem. Specifically, we fit a GNN to the network throughput and use the gradients of this GNN to iteratively update the locations of the network nodes. Besides, we show that an expressive GNN has the capacity to approximate both the function value and the gradients of a multivariate permutation-invariant function, as a theoretic support to the proposed method. To further improve the throughput, we also study a hybrid node deployment method based on this approach. To train the desired GNN, we adopt a policy gradient algorithm to create datasets containing good training samples. Numerical experiments show that the proposed methods produce competitive results compared to the baselines.

Compared to a single UAV with limited sensing coverage and communication capability, multi-UAV cooperation is able to provide more effective sensing and transmission (S&T) services. Nevertheless, most existing works on multi-UAV sensing mainly focus on mutually exclusive task allocation and independent data transmission, which did not fully exploit the benefit of multi-UAV sensing and communication. Motivated by this, we propose a novel multi-UAV cooperative S&T scheme with replicated sensing task allocation. Although replicated task allocation may sound counter-intuitive, it can actually foster cooperative transmission among multiple UAVs and thus reduce the overall sensing mission completion time. To obtain the optimal task allocation and transmit power of the proposed scheme, a mission completion time minimization problem is formulated. To solve this problem, a necessary condition for replicated sensing task allocation is derived. For the cases of replicated sensing, the considered problem is transformed into a monotonic optimization and is solved by the generic Polyblock algorithm. To efficiently evaluate the mission completion time in each iteration of the Polyblock algorithm, new auxiliary variables are introduced to decouple the otherwise sophisticated joint optimization of transmission time and power. While for the degenerated case of non-replicated sensing, the closed-form expression of the optimal transmission time is derived, Comment: 32 pages, submitted to IEEE for possible publication

The performance of mobile edge computing (MEC) depends critically on the quality of the wireless channels. From this viewpoint, the recently advocated intelligent reflecting surface (IRS) technique that can proactively reconfigure wireless channels is anticipated to bring unprecedented performance gain to MEC. In this paper, the problem of network throughput optimization of an IRS-assisted multi-hop MEC network is investigated, in which the phase-shifts of the IRS and the resource allocation of the relays need to be jointly optimized. However, due to the coupling among the transmission links of different hops caused by the utilization of the IRS and the complicated multi-hop network topology, it is difficult to solve the considered problem by directly applying existing optimization techniques. Fortunately, by exploiting the underlying structure of the network topology and spectral graph theory, it is shown that the network throughput can be well approximated by the second smallest eigenvalue of the network Laplacian matrix. This key finding allows us to develop an effective iterative algorithm for solving the considered problem. Numerical simulations are performed to corroborate the effectiveness of the proposed scheme.

In federated learning (FL), reducing the communication overhead is one of the most critical challenges since the parameter server and the mobile devices share the training parameters over wireless links. With such consideration, we adopt the idea of SignSGD in which only the signs of the gradients are exchanged. Moreover, most of the existing works assume Channel State Information (CSI) available at both the mobile devices and the parameter server, and thus the mobile devices can adopt fixed transmission rates dictated by the channel capacity. In this work, only the parameter server side CSI is assumed, and channel capacity with outage is considered. In this case, an essential problem for the mobile devices is to select appropriate local processing and communication parameters (including the transmission rates) to achieve a desired balance between the overall learning performance and their energy consumption. Two optimization problems are formulated and solved, which optimize the learning performance given the energy consumption requirement, and vice versa. Furthermore, considering that the data may be distributed across the mobile devices in a highly uneven fashion in FL, a stochastic sign-based algorithm is proposed. Extensive simulations are performed to demonstrate the effectiveness of the proposed methods.

Federated learning (FL) has emerged as a prominent distributed learning paradigm. FL entails some pressing needs for developing novel parameter estimation approaches with theoretical guarantees of convergence, which are also communication efficient, differentially private and Byzantine resilient in the heterogeneous data distribution settings. Quantization-based SGD solvers have been widely adopted in FL and the recently proposed SIGNSGD with majority vote shows a promising direction. However, no existing methods enjoy all the aforementioned properties. In this paper, we propose an intuitively-simple yet theoretically-sound method based on SIGNSGD to bridge the gap. We present Stochastic-Sign SGD which utilizes novel stochastic-sign based gradient compressors enabling the aforementioned properties in a unified framework. We also present an error-feedback variant of the proposed Stochastic-Sign SGD which further improves the learning performance in FL. We test the proposed method with extensive experiments using deep neural networks on the MNIST dataset and the CIFAR-10 dataset. The experimental results corroborate the effectiveness of the proposed method.

The deployment of unmanned aerial vehicles (UAVs) is proliferating as they are effective, flexible and cost-efficient devices for a variety of applications ranging from natural disaster recovery to delivery of goods. We investigate a transmission mechanism aiming to improve the data rate between a base station (BS) and a user equipment through deploying multiple relaying UAVs. We consider the effect of interference, which is incurred by the nodes of another established communication network. Our primary goal is to design the 3D trajectories and power allocation for the UAVs to maximize the data flow while the interference constraint is met. The UAVs can reconfigure their locations to evade the unintended/intended interference caused by reckless/smart interferers. We also consider the scenario in which smart jammers chase the UAVs to degrade the communication quality. In this case, we investigate the problem from the perspective of both UAV network and smart jammers. An alternating-maximization approach is proposed to address the joint 3D trajectory design and power allocation problem. We handle the 3D trajectory design by resorting to spectral graph theory and subsequently address the power allocation through convex optimization techniques. Finally, we demonstrate the efficacy of our proposed method through simulations., Comment: 12 pages, 13 figures. arXiv admin note: substantial text overlap with arXiv:1904.07781

The use of the unmanned aerial vehicle (UAV) has been foreseen as a promising technology for the next generation communication networks. Since there are no regulations for UAVs deployment yet, most likely they form a network in coexistence with an already existed network. In this work, we consider a transmission mechanism that aims to improve the data rate between a terrestrial base station (BS) and user equipment (UE) through deploying multiple UAVs relaying the desired data flow. Considering the coexistence of this network with other established communication networks, we take into account the effect of interference, which is incurred by the existing nodes. Our primary goal is to optimize the three-dimensional (3D) trajectories and power allocation for the relaying UAVs to maximize the data flow while keeping the interference to existing nodes below a predefined threshold. An alternating-maximization strategy is proposed to solve the joint 3D trajectory design and power allocation for the relaying UAVs. To this end, we handle the information exchange within the network by resorting to spectral graph theory and subsequently address the power allocation through convex optimization techniques. Simulation results show that our approach can considerably improve the information flow while the interference threshold constraint is met., Comment: 6 pages, 5 figures

The recent advances in sensor technologies and smart devices enable the collaborative collection of a sheer volume of data from multiple information sources. As a promising tool to efficiently extract useful information from such big data, machine learning has been pushed to the forefront and seen great success in a wide range of relevant areas such as computer vision, health care, and financial market analysis. To accommodate the large volume of data, there is a surge of interest in the design of distributed machine learning, among which stochastic gradient descent (SGD) is one of the mostly adopted methods. Nonetheless, distributed machine learning methods may be vulnerable to Byzantine attack, in which the adversary can deliberately share falsified information to disrupt the intended machine learning procedures. Therefore, two asynchronous Byzantine tolerant SGD algorithms are proposed in this work, in which the honest collaborative workers are assumed to store the model parameters derived from their own local data and use them as the ground truth. The proposed algorithms can deal with an arbitrary number of Byzantine attackers and are provably convergent. Simulation results based on a real-world dataset are presented to verify the theoretical results and demonstrate the effectiveness of the proposed algorithms.

Aerial base station (ABS) is a promising solution for public safety as it can be deployed in coexistence with cellular networks to form a temporary communication network. However, the interference from the primary cellular network may severely degrade the performance of an ABS network. With this consideration, an adaptive dynamic interference avoidance scheme is proposed in this work for ABSs coexisting with a primary network. In the proposed scheme, the mobile ABSs can reconfigure their locations to mitigate the interference from the primary network, so as to better relay the data from the designated source(s) to destination(s). To this end, the single/multi-commodity maximum flow problems are formulated and the weighted Cheeger constant is adopted as a criterion to improve the maximum flow of the ABS network. In addition, a distributed algorithm is proposed to compute the optimal ABS moving directions. Moreover, the trade-off between the maximum flow and the shortest path trajectories is investigated and an energy-efficient approach is developed as well. Simulation results show that the proposed approach is effective in improving the maximum network flow and the energy-efficient approach can save up to 39% of the energy for the ABSs with marginal degradation in the maximum network flow., Comment: 9 pages, 13 figures, to be presented in Proc. IEEE INFOCOM 2019

While machine learning has achieved remarkable results in a wide variety of domains, the training of models often requires large datasets that may need to be collected from different individuals. As sensitive information may be contained in the individual's dataset, sharing training data may lead to severe privacy concerns. Therefore, there is a compelling need to develop privacy-aware machine learning methods, for which one effective approach is to leverage the generic framework of differential privacy. Considering that stochastic gradient descent (SGD) is one of the most commonly adopted methods for large-scale machine learning problems, a decentralized differentially private SGD algorithm is proposed in this work. Particularly, we focus on SGD without replacement due to its favorable structure for practical implementation. Both privacy and convergence analysis are provided for the proposed algorithm. Finally, extensive experiments are performed to demonstrate the effectiveness of the proposed method.