Search

This paper proposes a differential privacy decentralized zeroth-order gradient tracking optimization (DP-DZOGT) algorithm for solving optimization problems of decentralized systems, where the gradient information of the function is unknown. To address the challenge of unknown gradient information, a one-point zeroth-order gradient estimator (OPZOGE) is constructed, which can estimate the gradient based on the function value and guide the update of decision variables. Additionally, to prevent privacy leakage of agents, random noise is introduced into both the state and the gradient of the agents, which effectively enhances the level of privacy protection. The linear convergence of the proposed DP-DZOGT under a fixed step size can be guaranteed. Moreover, it has been applied to the fields of smart grid (SG) and decentralized federated learning (DFL). Finally, the effectiveness of the algorithm is validated through three numerical simulations. • This paper proposes a differential privacy decentralized zeroth-order gradient tracking optimization algorithm. • The proposed algorithm not only protects data privacy but also effectively improves convergence speed. • This paper explores the practical applications of DP-DZOGT in the domains of smart grids and decentralized federated learning. [ABSTRACT FROM AUTHOR]

Chemotaxis is a directed cell movement in response to external chemical stimuli. In this paper, we propose a simple model for the origin of chemotaxis – namely how a directed movement in response to an external chemical signal may occur based on purely reaction–diffusion equations reflecting inner working of the cells. The model is inspired by the well-studied role of the rho-GTPase Cdc42 regulator of cell polarity, in particular in yeast cells. We analyse several versions of the model to better understand its analytic properties and prove global regularity in one and two dimensions. Using computer simulations, we demonstrate that in the framework of this model, at least in certain parameter regimes, the speed of the directed movement appears to be proportional to the size of the gradient of signalling chemical. This coincides with the form of the chemical drift in the most studied mean field model of chemotaxis, the Keller–Segel equation. [ABSTRACT FROM AUTHOR]

To prove that the hybrid teaching mode of ideological and political theory courses in colleges and universities is significantly helpful in improving the teaching effect of Civics and Political Science, this study builds a basic framework of the hybrid teaching mode of the course by using the smart teaching platform starting from the accurate teaching mode of information technology. For the user tracking of the smart teaching platform, the gradient tracking multi-user detection algorithm based on the multi-step proposed Newton method is used to realize the user data analysis and the convergence analysis is conducted for the GDGP-MUD algorithm and MSQN-MUD algorithm. The results show that the maximum average response time is 460ms when the number of concurrent users of the smart teaching platform is 350-400. The average performance of the experimental class under the comparative analysis is 11.41 points higher than that of the control class.

Chemotaxis is a directed cell movement in response to external chemical stimuli. In this paper, we propose a simple model for the origin of chemotaxis – namely how a directed movement in response to an external chemical signal may occur based on purely reaction–diffusion equations reflecting inner working of the cells. The model is inspired by the well-studied role of the rho-GTPase Cdc42 regulator of cell polarity, in particular in yeast cells. We analyse several versions of the model to better understand its analytic properties and prove global regularity in one and two dimensions. Using computer simulations, we demonstrate that in the framework of this model, at least in certain parameter regimes, the speed of the directed movement appears to be proportional to the size of the gradient of signalling chemical. This coincides with the form of the chemical drift in the most studied mean field model of chemotaxis, the Keller–Segel equation.

This paper considers the decentralized convex optimization problem, which has a wide range of applications in large-scale machine learning, sensor networks, and control theory. We propose novel algorithms that achieve optimal computation complexity and near optimal communication complexity. Our theoretical results give affirmative answers to the open problem on whether there exists an algorithm that can achieve a communication complexity (nearly) matching the lower bound depending on the global condition number instead of the local one. Furthermore, the linear convergence of our algorithms only depends on the strong convexity of global objective and it does not require the local functions to be convex. The design of our methods relies on a novel integration of well-known techniques including Nesterov's acceleration, multi-consensus and gradient-tracking. Empirical studies show the outperformance of our methods for machine learning applications. [ABSTRACT FROM AUTHOR]

Emerging distributed applications recently boosted the development of decentralized machine learning, especially in IoT and edge computing fields. In real-world scenarios, the common problems of non-convexity and data heterogeneity result in inefficiency, performance degradation, and development stagnation. The bulk of studies concentrate on one of the issues mentioned above without having a more general framework that has been proven optimal. To this end, we propose a unified paradigm called UMP, which comprises two algorithms D-SUM and GT-DSUM based on the momentum technique with decentralized stochastic gradient descent (SGD). The former provides a convergence guarantee for general non-convex objectives, while the latter is extended by introducing gradient tracking, which estimates the global optimization direction to mitigate data heterogeneity (i.e. , distribution drift). We can cover most momentum-based variants based on the classical heavy ball or Nesterov's acceleration with different parameters in UMP. In theory, we rigorously provide the convergence analysis of these two approaches for non-convex objectives and conduct extensive experiments, demonstrating a significant improvement in model accuracy up to 57.6% compared to other methods in practice. [ABSTRACT FROM AUTHOR]

In this article, we concentrate on dealing with the distributed optimization problem over a directed network, where each unit possesses its own convex cost function and the principal target is to minimize a global cost function (formulated by the average of all local cost functions) while obeying the network connectivity structure. Most of the existing methods, such as push-sum strategy, have eliminated the unbalancedness induced by the directed network via utilizing column-stochastic weights, which may be infeasible if the distributed implementation requires each unit to gain access to (at least) its out-degree information. In contrast, to be suitable for the directed networks with row-stochastic weights, we propose a new directed distributed Nesterov-like gradient tracking algorithm, named as D-DNGT, that incorporates the gradient tracking into the distributed Nesterov method with momentum terms and employs nonuniform step-sizes. D-DNGT extends a number of outstanding consensus algorithms over strongly connected directed networks. The implementation of D-DNGT is straightforward if each unit locally chooses a suitable step-size and privately regulates the weights on information that acquires from in-neighbors. If the largest step-size and the maximum momentum coefficient are positive and small sufficiently, we can prove that D-DNGT converges linearly to the optimal solution provided that the cost functions are smooth and strongly convex. We provide numerical experiments to confirm the findings in this article and contrast D-DNGT with recently proposed distributed optimization approaches. [ABSTRACT FROM AUTHOR]

This article considers a problem of solving the optimal solution of the sum of locally convex cost functions over an undirected network. Each local convex cost function in the network is accessed only by each unit. To be able to reduce the amount of computation and get the desired result in an accelerated way, we put forward a fresh accelerated decentralized event-triggered algorithm, named as A-DETA, for the optimization problem. A-DETA combines gradient tracking and two momentum accelerated terms, adopts nonuniform step-sizes and emphasizes that each unit interacts with neighboring units independently only at the sampling time triggered by the event. On the premise of assuming the smoothness and strong convexity of the cost function, it is proved that A-DETA can obtain the exact optimal solution linearly in the event of sufficiently small positive step-size and momentum coefficient. Moreover, an explicit linear convergence speed is definitely shown. Finally, extensive simulation example validates the usability of A-DETA. [ABSTRACT FROM AUTHOR]

There is growing interest in large-scale machine learning and optimization over decentralized networks, e.g. in the context of multi-agent learning and federated learning. Due to the imminent need to alleviate the communication burden, the investigation of communication- efficient distributed optimization algorithms -- particularly for empirical risk minimization -- has flourished in recent years. A large fraction of these algorithms have been developed for the master/slave setting, relying on the presence of a central parameter server that can communicate with all agents. This paper focuses on distributed optimization over networks, or decentralized optimization, where each agent is only allowed to aggregate information from its neighbors over a network (namely, no centralized coordination is present). By properly adjusting the global gradient estimate via local averaging in conjunction with proper correction, we develop a communication-efficient approximate Newton-type method, called Network-DANE, which generalizes DANE to accommodate decentralized scenarios. Our key ideas can be applied, in a systematic manner, to obtain decentralized versions of other master/slave distributed algorithms. A notable development is Network-SVRG/SARAH, which employs variance reduction at each agent to further accelerate local computation. We establish linear convergence of Network-DANE and Network-SVRG for strongly convex losses, and Network-SARAH for quadratic losses, which shed light on the impacts of data homogeneity, network connectivity, and local averaging upon the rate of convergence. We further extend Network-DANE to composite optimization by allowing a nonsmooth penalty term. Numerical evidence is provided to demonstrate the appealing performance of our algorithms over competitive baselines, in terms of both communication and computation efficiency. Our work suggests that by performing a judiciously chosen amount of local communication and computation per iteration, the overall efficiency can be substantially improved. [ABSTRACT FROM AUTHOR]