Search

This work investigates the stabilization problem of uncertain stochastic Markovian jump systems (MJSs) under communication constraints. To reduce the bandwidth usage, a discrete-time Markovian chain is employed to implement the stochastic communication protocol (SCP) scheduling of the sensor nodes, by which only one sensor node is chosen to access the network at each transmission instant. Moreover, due to the effect of amplitude attenuation, time delay, and random interference/noise, the transmission may be inevitably subject to the Rice fading phenomenon. All of these constraints make the controller only receive the fading signal from one activated sensor node at each instant. A merge approach is first used to deal with two Markovian chains; meanwhile, a compensator is designed to provide available information for the controller. By a compensator and mode-based sliding-mode controller, the resulting closed-loop system is ensured to be input-to-state stable in probability (ISSiP), and the quasisliding mode is attained. Moreover, an iteration optimizing algorithm is provided to reduce the convergence domain around the sliding surface via searching a desirable sliding gain, which constitutes an effective GA-based sliding-mode control strategy. Finally, the proposed control scheme is verified via the simulation results.

In this article, the problem of H∞ codesign for nonlinear control systems with unmatched uncertainties and adjustable parameters is investigated. The main purpose is to solve the adjustable parameters and H∞ controller simultaneously so that better robust control performance can be achieved. By introducing a bounded function and defining a special cost function, the problem of solving the Hamilton-Jacobi-Isaacs equation is transformed into an optimization problem with nonlinear inequality constraints. Based on the sum of squares technique, a novel policy iteration algorithm is proposed to solve the problem of the H∞ codesign. Moreover, one modified algorithm for optimizing the robust performance index is given. The convergence and the performance improvement of new iteration policy algorithms are proved. Simulation results are presented to demonstrate the effectiveness of the proposed algorithms.

Mobile Deep Learning (MDL) has emerged as a privacy-preserving learning paradigm for mobile devices. This paradigm offers unique features such as privacy preservation, continual learning and low-latency inference to the building of personal mobile sensing applications. However, squeezing Deep Learning to mobile devices is extremely challenging due to resource constraint. Traditional Deep Neural Networks (DNNs) are usually over-parametered, hence incurring huge resource overhead for on-device learning. In this paper, we present a novel on-device deep learning framework named MDLdroidLite that transforms traditional DNNs into resource-efficient model structures for on-device learning. To minimize resource overhead, we propose a novel Release-and-Inhibit Control (RIC) approach based on Model Predictive Control theory to efficiently grow DNNs from tiny to backbone. We also design a gate-based fast adaptation mechanism for channel-level knowledge transformation to quickly adapt new-born neurons with existing neurons, enabling safe parameter adaptation and fast convergence for on-device training. Our evaluations show that MDLdroidLite boosts on-device training on various PMS datasets with 28x to 50x less model parameters, 4x to 10x less floating number operations than the state-of-the-art model structures while keeping the same accuracy level.

Convolutional transform learning (CTL), learning filters by minimizing the data fidelity loss function in an unsupervised way, is becoming very pervasive, resulting from keeping the best of both worlds: the benefit of unsupervised learning and the success of the convolutional neural network. There have been growing interests in developing efficient CTL algorithms. However, developing a convergent and accelerated CTL algorithm with accurate representations simultaneously with proper sparsity is an open problem. This article presents a new CTL framework with a log regularizer that can not only obtain accurate representations but also yield strong sparsity. To efficiently address our nonconvex composite optimization, we propose to employ the proximal difference of the convex algorithm (PDCA) which relies on decomposing the nonconvex regularizer into the difference of two convex parts and then optimizes the convex subproblems. Furthermore, we introduce the extrapolation technology to accelerate the algorithm, leading to a fast and efficient CTL algorithm. In particular, we provide a rigorous convergence analysis for the proposed algorithm under the accelerated PDCA. The experimental results demonstrate that the proposed algorithm can converge more stably to desirable solutions with lower approximation error and simultaneously with stronger sparsity and, thus, learn filters efficiently. Meanwhile, the convergence speed is faster than the existing CTL algorithms.

Because of COVID-19, factories are facing many difficulties, such as shortage of workers and social alienation. How to improve production performance under limited labor resources is an urgent problem for global manufacturing factories. This work studies an energy-efficient job-shop scheduling problem with limited workers. Those workers can have multiskills. A many-objective model with five objectives, that is: 1) makespan; 2) total tardiness; 3) total idle time; 4) total worker cost; and 5) total energy, is built. To solve this many-objective optimization problem (MaOP), a novel fitness evaluation mechanism (FEM) based on fuzzy correlation entropy (FCE) is adopted. Two construction methods for reference points are proposed to build the bridge between MaOP and a fuzzy set. Based on FCE and cluster methods, an environmental selection mechanism (ESM) is proposed to achieve a balance between solution convergence and diversity. With the proposed FEM and ESM, two many-objective evolutionary algorithms are proposed to solve MaOP. The effect of FCE-based FEM and ESM on the performance of algorithms is verified via experiments. The proposed algorithms are compared with four well-known peers to test their performance. The extensive experimental results show that they are very competitive for the considered many-objective scheduling problem.

The security and economy of multienergy systems (MESs) are directly threatened by the potential cyberattacks. It is of great importance to investigate the effects of cyberattacks, e.g., denial of service (DoS) attacks, on distributed energy management algorithms. To this end, this article focuses on exploring how the frequency and the time of duration of DoS attacks influence the behavior of Newton-Raphson-based distributed energy management (NRBDEM) algorithm for MES and in which condition the optimal operations can still be obtained. First, a switched NRBDEM algorithm is presented, which is composed of the normal operation mode and the attack mode. In the attack mode, the attackers are able to change the communication structure at will and make it unconnected to destroy the convergence of the switched NRBDEM algorithm. Then, by making use of automatons to generate the hybrid time domain, the switched NRBDEM algorithm is further modeled and formulated as a hybrid dynamical system, which provides a mathematical model for the subsequent convergence analysis. Therein, the generated hybrid time domain satisfies the average dwell-time constraint and time-ratio constraint to limit the persistent attacks. Furthermore, we analyze the restrained conditions for persistent attacks, under which the optimality and convergence of the switched NRBDEM algorithm can be guaranteed still. Finally, simulation results demonstrate the effectiveness of the proposed method.

We study decentralized non-convex finite-sum minimization problems described over a network of nodes, where each node possesses a local batch of data samples. In this context, we analyze a single-timescale randomized incremental gradient method, called GT-SAGA. GT-SAGA is computationally efficient as it evaluates one component gradient per node per iteration and achieves provably fast and robust performance by leveraging node-level variance reduction and network-level gradient tracking. For general smooth non-convex problems, we show the almost sure and mean-squared convergence of GT-SAGA to a first-order stationary point and further describe regimes of practical significance where it outperforms the existing approaches and achieves a network topology-independent iteration complexity respectively. When the global function satisfies the Polyak-Lojaciewisz condition, we show that GT-SAGA exhibits linear convergence to an optimal solution in expectation and describe regimes of practical interest where the performance is network topology-independent and improves upon the existing methods. Numerical experiments are included to highlight the main convergence aspects of GT-SAGA in non-convex settings.

Eigendecomposition of symmetric matrices is at the heart of many computer vision algorithms. However, the derivatives of the eigenvectors tend to be numerically unstable, whether using the SVD to compute them analytically or using the Power Iteration (PI) method to approximate them. This instability arises in the presence of eigenvalues that are close to each other. This makes integrating eigendecomposition into deep networks difficult and often results in poor convergence, particularly when dealing with large matrices. While this can be mitigated by partitioning the data into small arbitrary groups, doing so has no theoretical basis and makes it impossible to exploit the full power of eigendecomposition. In previous work, we mitigated this using SVD during the forward pass and PI to compute the gradients during the backward pass. However, the iterative deflation procedure required to compute multiple eigenvectors using PI tends to accumulate errors and yield inaccurate gradients. Here, we show that the Taylor expansion of the SVD gradient is theoretically equivalent to the gradient obtained using PI without relying in practice on an iterative process and thus yields more accurate gradients. We demonstrate the benefits of this increased accuracy for image classification and style transfer., Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) PREPRINT 2021

A correlation between fuzzy logic systems (FLS) and zeroing neural networks (ZNN) design is investigated. It is shown that the gain parameter included in ZNN design can be dynamically adjusted over time by means of an appropriate value derived as the output of a properly defined FLS which includes appropriately defined membership functions and fuzzy logic rules. Dynamical systems which are applicable to time-varying rank-deficient matrices are proposed. Convergence properties are investigated and illustrative simulation experiments are performed. Presented simulation experiments confirm the superiority of the FLS proposed in the paper with respect to previously proposed FLS for dynamic adjustment of gain parameters. Furthermore, the superiority of the FLS-based ZNN model over the corresponding ZNN models based on the classical approach in defining the varying-gain parameter is demonstrated.

In this article, we present a novel matrix-factorization-based model, labeled here as Vicinal vertex allocated matrix factorization (VVAMo), for uncovering clusters in network data. Different from the past related efforts of network clustering, which consider the edge structure, vertex features, or both in their design, the proposed model includes the additional detail on vertex inclinations with respect to topology and features into the learning. In particular, by taking the latent preferences between vicinal vertices into consideration, VVAMo is then able to uncover network clusters composed of proximal vertices that share analogous inclinations, and correspondingly high structural and feature correlations. To ensure such clusters are effectively uncovered, we propose a unified likelihood function for VVAMo and derive an alternating algorithm for optimizing the proposed function. Subsequently, we provide the theoretical analysis of VVAMo, including the convergence proof and computational complexity analysis. To investigate the effectiveness of the proposed model, a comprehensive empirical study of VVAMo is conducted using extensive commonly used realistic network datasets. The results obtained show that VVAMo attained superior performances over existing classical and state-of-the-art approaches. AI Singapore National Research Foundation (NRF) This work was supported in part by the National Research Foundation, Singapore under its AI Singapore Programme (AISG) under Award AISG-RP-2018-004; and in part by the Data Science Artificial Intelligence Research Centre, Nanyang Technological University, and the Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU), which is a collaboration between Singapore Telecommunications Limited (Singtel) and Nanyang Technological University (NTU) that is funded by the Singapore Government through the Industry Alignment Fund—Industry Collaboration Projects Grant.

In this article, a new learning-based time-varying impedance controller is proposed and tested to facilitate an autonomous physical human-robot interaction (pHRI). Novel adaptation laws are formulated for online adjustment of robot impedance based on human behavior. Two other sets of update rules are defined for intelligent coping with the robot's structured and unstructured uncertainties. These rules ensure stability via Lyapunov's theorem and provide uniform ultimate boundedness (UUB) of the closed-loop system's response, without a need for HRI force/torque measurement. Accordingly, the convergence of response signals, including errors in tracking, online impedance learning, robot parameter adaptation, and controller gain variation, is proven to operate in a bounded region (compact set) in the presence of robot and human uncertainties and bounded disturbances. The performance of the developed intelligent impedance-varying control strategy is investigated through comprehensive experimental studies in a repetitive following task with a moving target.

This paper presents a simple control method for the fully automatic parking of an unmanned vehicle. This method is based on the switching control algorithm and backstepping theory. The fully automatic parking is hard to accomplish since it cannot converge to a specified norm, guarantee the convergence rate, and large uncertainties. Global exponential convergence to any prescribed norm bound (e-convergence) is guaranteed, and the convergence rate is explicitly given. For the design of steering laws, a method based on backstepping is proposed and analyzed in detail. The backstepping-based design of a multichain system is obtained. Moreover, an exponentially e-convergent control algorithm is adopted to guarantee both converge norm and convergence rate. Real road experiment results are presented and the results show the effectiveness of the control strategies.

In this paper, we consider stochastic master equations describing the evolution of a multi-qubit system interacting with electromagnetic fields undergoing continuous-time measurements. By considering multiple z-type (Pauli z matrix on different qubits) and x-type (Pauli x matrix on all qubits) measurements and one control Hamiltonian, we provide general conditions on the feedback controller and the control Hamiltonian ensuring almost sure exponential convergence to a predetermined Greenberger-Horne-Zeilinger (GHZ) state, which is assumed to be a common eigenstate of the measurement operators. We provide explicit expressions of feedback controllers satisfying such conditions. We also consider the case of only z-type measurements and multiple control Hamiltonians. We show that local stability in probability holds true, however, the reachability of a neighborhood of a predetermined GHZ state is not clear. In this case, we provide a heuristic discussion on some conditions which may ensure asymptotic convergence toward the target state. Finally, we demonstrate the effectiveness of our methodology for a three-qubit system through numerical simulations.

This paper investigates the event-triggered adaptive fuzzy fixed-time secure control problem for a class of nonlinear cyber-physical systems (CPSs) subject to unknown deception attacks and backlash-like hysteresis. Based on an improved fractional-order command filtered backstepping (FOCFB) method, fuzzy approximation technique, and Nussbaum gain technique, a novel adaptive practical fixed-time secure control scheme is proposed. Theoretical analysis proves that the fixed-time stability of the resulting control system and boundedness of all signals in the closed-loop system can be guaranteed by using the presented resilient controller. Especially, the given convergence time is independent of the initial states of the system and better system performance consisting of higher control accuracy and faster convergence rate can be ensured. Finally, a chemical reaction system is carried out to show the validity and superiority of the developed secure control scheme.

This paper proposes a novel high-performance deep learning framework for load recognition. The framework consists of a deep-shallow model and a fast backpropagation (FBP) algorithm. In the deep-shallow model, power-related wave patterns are perceived by a convolutional neural network (CNN), and feature statistics of the power consumption data are analyzed by a sparse feed-forward neural network. The new architecture improves model interpretability and prevents information loss problems in CNNs. The architecture also greatly boosts the convergence speed and significantly enhances the test set accuracy of a neural net-work. Compared with conventional CNN models utilized by many load recognition applications, the FBP algorithm that constitutes four sub-algorithms converges faster at the start of the training process and reduces at least 87.5% of the filter gradient compu-tations on average. The deep-shallow-fast model that combines the deep-shallow model and the FBP algorithm attains 97.62% accuracy on the test set in the load recognition task. To fully utilize the training data, a data augmentation technique is in-vented that transforms the voltage and current time series into an image-like 4-D tensor. Experiments illustrate that the proposed framework is much more accurate and converges considerably faster than the conventional CNN model that many deep-learning-based load recognition applications are based up-on.

Solving constrained multiobjective optimization problems brings great challenges to an evolutionary algorithm, since it simultaneously requires the optimization among several conflicting objective functions and the satisfaction of various constraints. Hence, how to adjust the tradeoff between objective functions and constraints is crucial. In this article, we propose a dynamic selection preference-assisted constrained multiobjective differential evolutionary (DE) algorithm. In our approach, the selection preference of each individual is suitably switching from the objective functions to constraints as the evolutionary process. To be specific, the information of objective function, without considering any constraints, is extracted based on Pareto dominance to maintain the convergence and diversity by exploring the feasible and infeasible regions; while the information of constraint is used based on constrained dominance principle to promote the feasibility. Then, the tradeoff in these two kinds of information is adjusted dynamically, by emphasizing the utilization of objective functions at the early stage and focusing on constraints at the latter stage. Furthermore, to generate the promising offspring, two DE operators with distinct characteristics are selected as components of the search algorithm. Experiments on four test suites including 56 benchmark problems indicate that the proposed method exhibits superior or at least competitive performance, in comparison with other well-established methods.

Previous research on finite-time control focuses on forcing a system state (vector) to converge within a certain time moment, regardless of how each state element converges. In the present work, we introduce a control problem with unique finite/fixed-time stability considerations, namely time-synchronized stability (TSS), where at the same time, all the system state elements converge to the origin, and fixed-TSS, where the upper bound of the synchronized settling time is invariant with any initial state. Accordingly, sufficient conditions for (fixed-) TSS are presented. On the basis of these formulations of the time-synchronized convergence property, the classical sign function, and also a norm-normalized sign function, are first revisited. Then in terms of this notion of TSS, we investigate their differences with applications in control system design for first-order systems (to illustrate the key concepts and outcomes), paying special attention to their convergence performance. It is found that while both these sign functions contribute to system stability, nevertheless an important result can be drawn that norm-normalized sign functions help a system to additionally achieve TSS. Furthermore, we propose a fixed-time-synchronized sliding-mode controller for second-order systems; and we also consider the important related matters of singularity avoidance there. Finally, numerical simulations are conducted to present the (fixed-) time-synchronized features attained; and further explorations of the merits of the proposed (fixed-) TSS are described.

Astrobots are robotic artifacts whose swarms are used in astrophysical studies to generate the map of the observable universe. These swarms have to be coordinated with respect to various desired observations. Such coordination\footnote{\z{A coordination sample of the astrobots may be found in \url{http://y2u.be/MpXWvpz4h00.}}} are so complicated that distributed swarm controllers cannot always coordinate enough astrobots to fulfill the minimum data desired to be obtained in the course of observations. Thus, a convergence verification is necessary to check the suitability of a coordination before its execution. However, a formal verification method does not exist for this purpose. In this paper, we instead use machine learning to predict the convergence of astrobots swarm. \z{As the first solution to this problem}, we propose a weighted $k$-NN-based algorithm which \rr{requires the initial status of a swarm as well as its observational targets to predict its convergence. Our algorithm learns to predict based on the coordination data obtained from previous coordination of the desired swarm. This method first generates a convergence probability for each astrobot based on a distance metric. Then, these probabilities are transformed to either a complete or an incomplete categorical result.} The method is applied to two typical swarms including 116 and 487 astrobots. It turns out that the correct prediction of successful coordination may be up to 80\% of overall predictions. Thus, these results witness the efficient accuracy of our predictive convergence analysis strategy.

This article investigates the adaptive fuzzy tracking control design for uncertain nonstrict-feedback nonlinear systems with time-varying constraints and asymmetric input saturations. It is known that time-varying output constraint, time-varying error constraints, and input saturations are commonly-seen issues in many practical engineering systems due to inherent physical limitations and performance requirements. To deal with these problems and improve the convergence rate of the control system, a novel fixed-time convergent adaptive fuzzy control scheme is proposed for the considered uncertain nonlinear systems via the time-varying barrier Lyapunov function (BLF) technique. Firstly, to deal with the unknown nonlinearities and uncertainties in the system dynamic model, fuzzy logic systems are utilized to approximate the unknown functions of the considered model. Then, the problem of asymmetric input amplitude and rate saturations is handled by constructing a unified smooth characterizing function and an innovative auxiliary design signal system. Subsequently, considering physical limitations and performance requirements, a novel time-varying BLF-based adaptive fuzzy backstepping control scheme is designed for the uncertain nonstrict-feedback nonlinear systems to realize superior tracking performances and keep the states staying in predefined time-varying compact regions during operations. Further, rigorous theoretical analyses have been conducted to show that the proposed control scheme achieves fixed-time convergence of all signals in the closed-loop control system. Finally, representative simulation results verify the effectiveness of the proposed control scheme.

The multi-target tracking (MTT) based on distributed fusion methods in an acoustic sensor array network (ASAN) is limited by the performance of measurement parameter estimators, such as the received signal strength (RSS) and the direction-of-arrival (DOA). For measurement parameters with low accuracy and resolution, the MTT may fail in subsequent steps, e.g., data association, because the loss of upstream information cannot be made up by downstream processing. Thus, we propose a new wideband MTT algorithm based on dynamic Bayesian network (DBN), which treats the ASAN as an overall extended array, and directly estimates the target states from the raw acoustic data. The DBN fuses the near-field model, the acoustic propagation model and the motion model. These sub-models can be optimized by each other, improving the final estimation. Also, for each sub-band, target signals and the precision parameters of the sensor noise are treated as hidden random variables. Based on this, the weight of each sub-band can be automatically adjusted according to the accurate hidden variables. Besides, the optimization problem of the posterior probability is transformed into a graphical model learning problem. Moreover, for non-conjugate models, a novel algorithm based on Laplace approximations (LAs) with Newton’s method (NM) is developed, i.e., DBN-LA-NM, bypassing data association. In addition, the corresponding CRLB and convergence conditions are derived. The numerical simulation results show that the proposed algorithm outperforms existing MTTs based on the near-field model in terms of accuracy, convergence and computational complexity. Field experiments further verify the feasibility of the proposed algorithm.

Electrochemical mechanisms in lithium-ion batteries (LIBs) pose a significant challenge in deriving models that are highly accurate, have low computational complexity, and enable real-time state and parameter estimation. In this article, we propose a machine learning model as an important building block of a physics-based ANCF-e model that was recently proposed for LIBs. This machine learning model is used to estimate nonlinear potentials, including the open-circuit potential, electrolyte potential, and lithium-intercalation overpotential. Such an estimation is shown to result in a much smaller computational complexity and therefore can enable real-time state and parameter estimation. Three different machine learning architectures are explored, including multilayer perceptron, radial basis function (RBF)-based neural networks, and support vector machines. The training of these machine learning models is carried out using current profiles obtained with an electric vehicle model from driving cycles as inputs and ANCF-e model-based outputs. The underlying ANCF-e model is validated both through a high-fidelity numerical approach, including COMSOL and an experimental test using commercial LIBs. Both validations are carried out under both constant current discharging and dynamic load cycles. The resulting performance using these machine learning models is compared using different metrics, including estimation errors, convergence rates, training time, and computational time. The results indicate that an RBF-based neural network leads to better estimation of the underlying potentials in LIBs and that all machine learning models require a computational time that is 95% smaller than a physics-based approach for this estimation.

An efficiency-oriented solution is theoretically a preferred choice to support the efficient operation of a system. Although some studies on the multi-manipulator system share the load of the control center by transforming the network topology, the whole system often suffers an increased communication burden. In this article, a multi-manipulator cooperative control scheme with improved communication efficiency is proposed to allocate limited communication resources reasonably. The entire control process is formulated from the perspective of game theory and finally evolved into a problem of finding a Nash equilibrium with time-varying parameters. Then, a neural network solver is designed to update the strategies of manipulators. Theoretical analysis supports the convergence and robustness of the solver. In addition, Zeno behavior does not occur under the domination of the control strategy. Finally, simulative results reveal that the proposed control strategy has advantages over traditional periodic control in communication.

The Binary Iterative Hard Thresholding (BIHT) algorithm is a popular reconstruction method for one-bit compressed sensing due to its simplicity and fast empirical convergence. There have been several works about BIHT but a theoretical understanding of the corresponding approximation error and convergence rate still remains open. This paper shows that the normalized version of BIHT (NBHIT) achieves an approximation error rate optimal up to logarithmic factors. More precisely, using $m$ one-bit measurements of an $s$-sparse vector $x$, we prove that the approximation error of NBIHT is of order $O \left(1 \over m \right)$ up to logarithmic factors, which matches the information-theoretic lower bound $\Omega \left(1 \over m \right)$ proved by Jacques, Laska, Boufounos, and Baraniuk in 2013. To our knowledge, this is the first theoretical analysis of a BIHT-type algorithm that explains the optimal rate of error decay empirically observed in the literature. This also makes NBIHT the first provable computationally-efficient one-bit compressed sensing algorithm that breaks the inverse square root error decay rate $O \left(1 \over m^{1/2} \right)$., Comment: Submitted to a journal

Recently, many real-world applications exploit multi-view data, which is collected from diverse domains or obtained from various feature extractors and reflect different properties or distributions of the data. In this work, a novel unsupervised multi-view framework is proposed to cluster such data. The proposed method, called Multi-View clustering with Adaptive Sparse Memberships and Weight Allocation (MVASM), pays more attention to constructing a common membership matrix with proper sparseness over different views and learns the centroid matrix and its corresponding weight of each view. Concretely, MVASM method attempts to learn a common and flexible sparse membership matrix to indicate the clustering, which explores the underlying consensus information of multiple views, and solves the multiple centroid matrices and weights to utilize the view-specific information and further modifies the above-mentioned membership matrix. In addition, the theoretical analysis, including the determination of the power exponent parameter, convergence analysis, and complexity analysis are also presented. Compared to the state-of-the-art methods, the proposed method improves the performance of clustering on different public datasets and demonstrates its reasonability and superiority.

Offspring generation plays an important role in evolutionary multiobjective optimization. However, generating promising candidate solutions effectively in high-dimensional spaces is particularly challenging. To address this issue, we propose an adaptive offspring generation method for large-scale multiobjective optimization. First, a preselection strategy is proposed to select a balanced parent population, and then these parent solutions are used to construct direction vectors in the decision spaces for reproducing promising offspring solutions. Specifically, two kinds of direction vectors are adaptively used to generate offspring solutions. The first kind takes advantage of the dominated solutions to generate offspring solutions toward the Pareto optimal set (PS) for convergence enhancement, while the other kind uses those nondominated solutions to spread the solutions over the PS for diversity maintenance. The proposed offspring generation method can be embedded in many existing multiobjective evolutionary algorithms (EAs) for large-scale multiobjective optimization. Experiments are conducted to reveal the mechanism of our proposed adaptive reproduction strategy and validate its effectiveness. Experimental results on some large-scale multiobjective optimization problems have demonstrated the competitive performance of our proposed algorithm in comparison with five state-of-the-art large-scale EAs.

This paper studies the problem of distributed optimization in multi-agent systems where each agent seeks to minimize the sum of all agents objective functions. Under the requirement of security, each of them needs to keep its objective function private from other agents and potential eavesdroppers. We first prove the impossibility of guaranteeing convergence and differential privacy simultaneously by perturbing states in exact distributed optimization algorithms. Motivated by this result, we design a completely distributed algorithm, which achieves differential privacy by perturbing both states and directions with decaying Laplace noise. Different from most of the existing works that require decaying stepsize to ensure convergence, we show that our algorithm converges in mean and almost surely even with a constant stepsize. In particular, we prove linear convergence in mean by only assuming that the sum of all cost functions is strongly convex. The R-linear convergence is proved under the assumption of Lipschitz gradients instead of that of bounded gradients. The optimal stepsize for fastest convergence rate is also established. Moreover, we describe the privacy properties and cha

This work considers the problem of resilient consensus, where stochastic values of trust between agents are available. Specifically, we derive a unified mathematical framework to characterize convergence, deviation of the consensus from the true consensus value, and expected convergence rate, when there exists additional information of trust between agents. We show that under certain conditions on the stochastic trust values and consensus protocol: First, almost sure convergence to a common limit value is possible even when malicious agents constitute more than half of the network connectivity; second, the deviation of the converged limit, from the case where there is no attack, i.e., the true consensus value, can be bounded with probability that approaches 1 exponentially; and third correct classification of malicious and legitimate agents can be attained in finite time almost surely. Furthermore, the expected convergence rate decays exponentially as a function of the quality of the trust observations between agents.

In this paper, we propose a deep reinforcement learning (DRL)-based algorithm to generate policies of Baseband Function (BBF) placement and routing. In order to explore the performance of the proposed algorithm in practical systems, the online scenario with the completely random requests is used in the simulation considering C-RAN and NG-RAN architectures. Besides, an Integer Linear Programming (ILP) model is formulated to generate the optimal solution as the benchmark. The simulation results show that DRL-based algorithm converges in a short time, and its performance closes to the optimal benchmark obtained by ILP in terms of latency and bandwidth for the online scenarios. In addition, the performance of the generated policies based on DRL is compared with a classic heuristic algorithm, i.e., first-fit algorithm. The performance of DRL-based algorithm is superior to the first-fit algorithm from above two perspectives. The fast convergence and the near-optimal performance prove that the DRL-based algorithm is a promising approach for the BBF placement and routing of RAN in 5G and Beyond.

In this article, a new self-learning parallel control method, which is based on adaptive dynamic programming (ADP) technique, is developed for solving the optimal control problem of discrete- time time-varying nonlinear systems. It aims to obtain an approximate optimal control law sequence and simultaneously guarantees the convergence of the value function. Establishing the time-varying artificial system by neural networks in a certain time-horizon, a control-sequence-improvement ADP algorithm is developed to obtain the control law sequence. For the first time, the criteria of the parallel execution are presented, such that the value function is proven to converge to a finite neighborhood of the optimal performance index function. Finally, numerical results and analysis are presented to demonstrate the effectiveness of the parallel control method.

This letter addresses the problem of steering a robotic vehicle along a geometric path specified with respect to a reference frame moving in three dimensions, termed the Moving Path Following (MPF) motion control problem. The MPF problem is solved for a large class of robotic vehicles that require a minimum positive forward speed to operate. This restriction poses additional challenges with respect to the control design, and is developed using geometric concepts, wherein the attitude control problem is formulated on the Special Orthogonal group SO(3). To this end, a MPF error model formulation is first derived that allows to exclude the conservative constraints on the initial position of the vehicle with respect to the reference path by enabling the explicit control of the progression of a virtual point moving along the reference path. The task of the MPF control law is then to steer the vehicle towards the moving path and to converge to the virtual point. Stability and convergence guarantees of the resulting closed-loop system are provided using the Input-to-State Stability concept. In particular, we show that the proposed controller is robust to imperfect tracking errors and external disturbances (e.g., autopilot errors and wind gusts). Simulation results are presented to illustrate the efficacy of the proposed MPF control law.

Distributed learning algorithms aim to leverage distributed and diverse data stored at users' devices to learn a global phenomena by performing training amongst participating devices and periodically aggregating their local models' parameters into a global model. Federated learning is a promising paradigm that allows for extending local training among the participant devices before aggregating the parameters, offering better communication efficiency. However, in the cases where the participants' data are strongly skewed (i.e., non-IID), the local models can overfit local data, leading to low performing global model. In this paper, we first show that a major cause of the performance drop is the weighted distance between the distribution over classes on users' devices and the global distribution. Then, to face this challenge, we leverage the edge computing paradigm to design a hierarchical learning system that performs Federated Gradient Descent on the user-edge layer and Federated Averaging on the edge-cloud layer. In this hierarchical architecture, we formalize and optimize this user-edge assignment problem such that edge-level data distributions turn to be similar (i.e., close to IID), which enhances the Federated Averaging performance. Our experiments on multiple real-world datasets show that the proposed optimized assignment is tractable and leads to faster convergence of models towards a better accuracy value. 2013 IEEE. Scopus

Optimal performance is extremely important for hypersonic flight control. Different from the most existing methodologies which only consider basic control performance including stability, robustness and transient performance, this article deals with the design of nearly optimal tracking controllers for hypersonic flight vehicles (HFVs). Firstly, main controllers are developed for the velocity subsystem and the altitude subsystem of HFVs via concise fuzzy approximations. Then, optimal controllers are nearly implemented utilizing single-network adaptive critic design. Moreover, the stability of closed-loop systems and the convergence of optimal controllers are theoretically proved. Finally, compared simulation results are given to verify the superiority. The special contribution is the application of low-complex control structure owing to the critic-only network and advanced learning laws developed for fuzzy approximations, which is expected to guarantee satisfied real-time performance.

As a critical infrastructure of urban water systems, sewer networks suffer from serious corrosion and odor problems, which can be controlled by chemical dosing. It is a challenging task to optimize chemical distribution in such a hybrid system with continuous hydraulic flow, discrete pump operations, and dynamic constraints. In this article, we study real-time control of multiple pumps to achieve the desired chemical distribution in a sewer network. A novel hybrid optimization approach is developed, which involves an event-triggered scheme triggered by predicting proper pumping events at uncontrolled pumping stations, and an improved nature-inspired elephant herding optimization (iEHO) algorithm for scheduling pumping at controllable pumping stations. The proposed method is validated through simulation studies of a real-life sewer network using real measured data. Our strategy significantly improves chemical distribution with reduced costs, despite an astronomic searching space. The iEHO algorithm outperforms the genetic algorithm in terms of the quality of solutions and convergence efficiency.

Optimization problems are essential for a broad spectrum of applications, and have been extensively studied in many aspects. Most existing literature on optimization algorithms does not consider systems that involve parameter uncertainties. This paper studies a class of stochastic adaptive optimization problems in which identification of unknown parameters and search of the optimal solutions must be performed simultaneously. Due to a fundamental conflict between parameter identifiability and optimality in such problems, we introduce a method of adding stochastic dither signals into the system, which provides sufficient excitation for estimating the unknown parameters, leading to adaptive optimization algorithms. Joint identification and optimization algorithms are developed and their simultaneous convergence properties of parameter estimation and optimization variable updates are proved. Under both noise-free and noisy observations, the corresponding convergence rates are established. The main results of this paper reveal certain fundamental relationships and trade-off among updating step sizes, dither magnitudes, parameter estimation errors, optimization accuracy, and convergence rates. Simulation case studies are used to illustrate the adaptive optimization algorithms and their main properties.

This article is concerned with the design of a super-twisting algorithm (STA) based sliding mode controller for permanent magnet synchronous motor (PMSM) speed regulation system under the self-triggered mechanism. By using the strict Lyapunov function approach, it is shown that the tracking error converges to an ultimate domain within the finite-time sense under the proposed self-triggered STA. A feasible self-triggered strategy is designed for both cases with and without external perturbation. Moreover, a nonlinear optimization problem is formulated in terms of the tradeoff between the ultimate domain and the communication burden. The optimized STA gains are obtained by solving the above-formulated optimization problem via a particle swarm optimization algorithm. Finally, the applicability of the proposed self-triggered STA for PMSM is verified by simulation and experiment results.

Road extraction from very high resolution (VHR) satellite images is one of the most important topics in the field of remote sensing. In this letter, we propose an efficient nonlocal LinkNet with nonlocal blocks (NLBs) that can grasp relations between global features. This enables each spatial feature point to refer to all other contextual information and results in more accurate road segmentation. In detail, our single model without any postprocessing like conditional random field (CRF) refinement performed better than any other published state-of-the-art ensemble model in the official DeepGlobe Challenge. Moreover, our nonlocal LinkNet (NL-LinkNet) beat the D-LinkNet, the winner of the DeepGlobe challenge (Demir et al., 2018), with 43% less parameters, less giga floating-point operations per seconds (GFLOPs), and shorter training convergence time. We also present empirical analyses on the proper usages of NLBs for the baseline model.

Hyperspectral images (HSIs) are used in a large number of real-world applications. HSI classification (HSIC) is a challenging task due to high interclass similarity, high intraclass variability, overlapping, and nested regions. The 2-D convolutional neural network (CNN) is a viable classification approach since HSIC depends on both spectral-spatial information. The 3-D CNN is a good alternative for improving the accuracy of HSIC, but it can be computationally intensive due to the volume and spectral dimensions of HSI. Furthermore, these models may fail to extract quality feature maps and underperform over the regions having similar textures. This work proposes a 3-D CNN model that utilizes both spatial-spectral feature maps to improve the performance of HSIC. For this purpose, the HSI cube is first divided into small overlapping 3-D patches, which are processed to generate 3-D feature maps using a 3-D kernel function over multiple contiguous bands of the spectral information in a computationally efficient way. In brief, our end-to-end trained model requires fewer parameters to significantly reduce the convergence time while providing better accuracy than existing models. The results are further compared with several state-of-the-art 2-D/3-D CNN models, demonstrating remarkable performance both in terms of accuracy and computational time.

Hashing techniques are in great demand for a wide range of real-world applications such as image retrieval and network compression. Nevertheless, existing approaches could hardly guarantee a satisfactory performance with the extremely low-bit (e.g., 4-bit) hash codes due to the severe information loss and the shrink of the discrete solution space. In this article, we propose a novel Collaborative Learning strategy that is tailored for generating high-quality low-bit hash codes. The core idea is to jointly distill bit-specific and informative representations for a group of pre-defined code lengths. The learning of short hash codes among the group can benefit from the manifold shared with other long codes, where multiple views from different hash codes provide the supplementary guidance and regularization, making the convergence faster and more stable. To achieve that, an asymmetric hashing framework with two variants of multi-head embedding structures is derived, termed as Multi-head Asymmetric Hashing (MAH), leading to great efficiency of training and querying. Extensive experiments on three benchmark datasets have been conducted to verify the superiority of the proposed MAH, and have shown that the 8-bit hash codes generated by MAH achieve 94.3 percent of the MAP 1 1. Mean Average Precision (MAP) score on the CIFAR-10 dataset, which significantly surpasses the performance of the 48-bit codes by the state-of-the-arts in image retrieval tasks.

This paper is aimed at realizing cooperative learning for networked multi-agent systems subject to uncertain nonlinear dynamics and switching topologies. A distributed control protocol is proposed by integrating the nearest neighbor rules and iterative updating rules. Thanks to cooperative learning, all agents can be ensured to track any prescribed reference robustly over any finite interval, regardless of nonidentical locally Lipschitz nonlinearities of agents, initial state shifts, and external disturbances. Moreover, a convergence analysis approach to cooperative learning is given by exploring the properties for the products of stochastic matrices that are associated with switching digraphs.

In this paper, we study the problem of federated learning over a wireless channel with user sampling, modeled by a fading multiple access channel, subject to central and local differential privacy (DP/LDP) constraints. It has been shown that the superposition nature of the wireless channel provides a dual benefit of bandwidth efficient gradient aggregation, in conjunction with strong DP guarantees for the users. Specifically, the central DP privacy leakage has been shown to scale as $\mathcal {O}(1/K^{1/2})$ , where $K$ is the number of users. It has also been shown that user sampling coupled with orthogonal transmission can enhance the central DP privacy leakage with the same scaling behavior. In this work, we show that, by jointly incorporating both wireless aggregation and user sampling, one can obtain even stronger privacy guarantees. We propose a private wireless gradient aggregation scheme, which relies on independently randomized participation decisions by each user. The central DP leakage of our proposed scheme scales as $\mathcal {O}(1/K^{3/4})$ . In addition, we show that LDP is also boosted by user sampling. We also present analysis for the convergence rate of the proposed scheme and study the tradeoffs between wireless resources, convergence, and privacy theoretically and empirically for two scenarios when the number of sampled participants are $(a)$ known, or $(b)$ unknown at the parameter server.

In this letter, we consider the problem of privacy preservation in the static average consensus problem. This problem normally is solved by proposing privacy preservation augmentations for the popular first order Laplacian-based algorithm. These mechanisms however come with computational overhead, may need coordination among the agents to choose their parameters and also alter the transient response of the algorithm. In this letter, we show that an alternative iterative algorithm that is proposed in the literature in the context of dynamic average consensus problem has intrinsic privacy preservation and can be used as a privacy preserving algorithm that yields the same performance behavior as the well-known Laplacian consensus algorithm but without the overheads that come with the existing privacy preservation methods.

A social cost minimization problem is addressed in this article. In the considered problem, a network of agents work collaboratively to minimize the social cost function, which is defined as the sum of the agents’ local objective functions. The engaged agents are supposed to be equipped with an undirected and connected communication graph. Different from most of the existing works, the social cost function in the considered problem is allowed to be nonconvex and possibly admits local extrema. To avoid local extrema and achieve the global minimization of the social cost function, an extremum-seeking-based approach is proposed by introducing a dynamic average consensus protocol to the sinusoidal-dither-signal-based extremum seeking scheme. The dynamic average consensus protocol is leveraged in the proposed extremum-seeker for information sharing and the sinusoidal probing signal is utilized for information extraction. For the avoidance of local extrema, the amplitude of the dither signal is designed to be adaptive. Through Lyapunov stability analysis, it is shown that the proposed method enables the decision variable to converge to a neighborhood of the global minimum point if the conditions on the network connectivity, the existence of unique global minimum and achievability of the global minimum are satisfied. The theoretical result is verified via simulating a numerical example.

This paper considers the problem of globally finite-time stabilizing a planar nonlinear system with a mis-matched unknown disturbance. First, a nonlinear integral dynamic is constructed to compensate the disturbance. For finite-time convergence, we use the system states and integral state to construct a new controller which not only is homogeneous with a negative degree but also contains a linear combination of states to handle the unmatched disturbance. The finite-time integral controller originally obtained for a linear planar system is appropriately scaled to regulate a nonlinear planar system within the framework of homogeneous domination. In addition, for a faster convergence speed for any initial condition, a dual-mode integral controller is introduced with better performances demonstrated by simulation studies.

In this article, we extend the theory of deterministic mean-field/aggregative games to multipopulation games. We consider a set of populations, each managed by a population coordinator (PC), of selfish agents playing a global noncooperative game, whose cost functions are affected by an aggregate term across all agents from all populations. In particular, we impose that the agents cannot exchange information between themselves directly; instead, only a PC can gather information on its own population and exchange local aggregate information with the neighboring PCs. To seek an equilibrium of the resulting (partial-information) game, we propose an iterative algorithm where each PC broadcasts a mean-field signal, namely, an estimate of the overall aggregative term, to its own population only. In turn, we let the local agents react with the best response and the PCs cooperate for estimating the aggregative term. Our main technical contributions are to cast the proposed scheme as a fixed-point iteration with errors, namely, the interconnection of a Krasnoselskij–Mann iteration and a linear consensus protocol, and, under a nonexpansiveness condition, to show convergence towards an ε -Nash equilibrium, where ε is inversely proportional to the population size.

This article considers the problem of event-triggered optimal control for discrete-time switched nonlinear systems with constrained control input. First, an event-triggered condition is given to make the closed-loop switched system asymptotically stable. Second, a novel method, event-triggered heuristic dynamic programming (ETHDP), is applied to derive the optimal control policy. Two neural networks (NNs) are utilized to approximate the value function and control law, respectively. When the event-triggered condition is violated, the weights of the two NNs are updated, which can decrease the networks calculation and transmission load notably. A proof of the convergence of the ETHDP is also carried out. Finally, the effectiveness of the proposed method is verified by an example.

Missing data is a critical pitfall in the investigation of remotely sensed displacement measurement because it prevents from a full understanding of the physical phenomenon under observation. In the sight of reconstructing incomplete displacement data, this letter presents a data-driven spatiotemporal gap-filling method, which is an extension of the expectation-maximization-empirical orthogonal function (EM-EOF) method. The presented method decomposes an augmented spatiotemporal covariance of a displacement time series into EOF modes and then selects the optimal set of EOF modes to reconstruct the time series. This selection is based on the cross-validation root-mean-square error and a confidence index associated with each eigenvalue. The estimated missing values are then iteratively updated until convergence. Results on displacement time series derived from cross correlation of Sentinel-2 optical images over Fox Glacier in New-Zealand's Alps show that the reconstruction accuracy is improved compared with the EM-EOF method. The proposed extension can tackle challenging cases, i.e., short time series with heterogeneous displacement behaviors corrupted by a large amount of missing data and noise.

The convergence of the waveform relaxation (WR) method is demonstrated for a class of circuits: Chains of identical and symmetrical passive subcircuits. The WR algorithm uses resistive coupling to implement the iteration. Every part is modeled as a symmetric and reciprocal linear two-port network. The iteration matrices of the WR operator are constructed for the Gauss-Jacobi and Gauss-Seidel relaxations in the Fourier domain. An upperbound estimate of the spectral radius of the WR operator is presented. It demonstrates the convergence of the method independently of the number of cascaded parts in the chain and the coupling resistance.

This article revisits a multiagent deployment problem where agents are restricted from requesting information from other agents as well as sending acknowledgments when information is received. These communication constraints relax the assumptions of instantaneous communication and synchronous actions by agents (request and response actions). In this article, we propose a fully asynchronous communication aware solution to the multiagent deployment problem that uses an event-triggered broadcasting strategy. Unlike all existing triggered solutions, our event-triggered broadcasting algorithm relies on agents to decide when to broadcast (push) information to others in the network without the need for a response from other agents. In addition, the proposed strategy determines how best to move when up-to-date information is unavailable and cannot be requested. The algorithm is capable of achieving similar levels of performance to that of a continuous or periodic strategy. Our solution is proven to achieve asymptotic convergence, and simulation results are provided to demonstrate that the proposed event-triggered broadcasting algorithm can achieve an adequate level of performance under the communication constraints.

Recently, many stochastic variance reduced alternating direction methods of multipliers (ADMMs) (e.g., SAG-ADMM and SVRG-ADMM) have made exciting progress such as linear convergence rate for strongly convex (SC) problems. However, their best-known convergence rate for non-strongly convex (non-SC) problems is $\mathcal {O}(1/T)$ O ( 1 / T ) as opposed to $\mathcal {O}(1/T^2)$ O ( 1 / T 2 ) of accelerated deterministic algorithms, where $T$ T is the number of iterations. Thus, there remains a gap in the convergence rates of existing stochastic ADMM and deterministic algorithms. To bridge this gap, we introduce a new momentum acceleration trick into stochastic variance reduced ADMM, and propose a novel accelerated SVRG-ADMM method (called ASVRG-ADMM) for the machine learning problems with the constraint $Ax + By = c$ A x + B y = c . Then we design a linearized proximal update rule and a simple proximal one for the two classes of ADMM-style problems with $B = \tau I$ B = τ I and $B\ne \tau I$ B ≠ τ I , respectively, where $I$ I is an identity matrix and $\tau$ τ is an arbitrary bounded constant. Note that our linearized proximal update rule can avoid solving sub-problems iteratively. Moreover, we prove that ASVRG-ADMM converges linearly for SC problems. In particular, ASVRG-ADMM improves the convergence rate from $\mathcal {O}(1/T)$ O ( 1 / T ) to $\mathcal {O}(1/T^2)$ O ( 1 / T 2 ) for non-SC problems. Finally, we apply ASVRG-ADMM to various machine learning problems, e.g., graph-guided fused Lasso, graph-guided logistic regression, graph-guided SVM, generalized graph-guided fused Lasso and multi-task learning, and show that ASVRG-ADMM consistently converges faster than the state-of-the-art methods.

This paper addresses the tradeoff problem between hit ratio and content quality in edge caching systems for multiuser adaptive bitrate streaming (ABS) services. A dynamic policy for cache decision and quality level selection for each ABS content during every cache cycle is proposed. Achieving this policy is NP-complete. For this, the considered problem is transformed into a nested multidimensional 0/1 knapsack optimization problem which is then resolved by a cooperative transfer learning-accelerated genetic algorithm. Performance evaluation demonstrates an adaptation of the proposed algorithm on various video stream popularity models in terms of algorithmic convergence and cache balancing.