Search

In recent years, multi-agent reinforcement learning techniques have been widely used and evolved in the field of artificial intelligence. However, traditional reinforcement learning methods have limitations such as long training time, large sample data requirements, and highly delayed rewards. Therefore, this paper systematically and specifically studies the MARL algorithm. Firstly, this paper uses Citespace software to visually analyze the existing literature on multi-agent reinforcement learning and briefly indicates the research hotspots and key research directions in this field. Secondly, the applications of traditional reinforcement learning algorithms under two task objects, namely single-agent and multi-agent systems, are described in detail. Then, the paper highlights the diverse applications, challenges, and corresponding solutions of MARL algorithmic techniques in the field of MAS. Finally, the paper points out future research directions based on the existing limitations of the algorithm. Through this paper, readers will gain a systematic and in-depth understanding of MARL algorithms and how they can be utilized to better address the various challenges posed by MAS. [ABSTRACT FROM AUTHOR]

This paper investigates the consensus problem of a class of high-dimensional nonlinear multi-agent systems. The agents are homogeneous and have coupled interactions with their nearby agents. Firstly, we study the consensus of this class of nonlinear multi-agent systems and present some consensus conditions based on the linearization technique. By transforming the consensus problem into a stability problem, we show that there exists a domain such that if all pairs of connected agents stay in the domain, the system will achieve consensus under the consensus conditions. Then, we propose a method to estimate the consensus domain since the accurate description of the domain is still challenging. Moreover, a method of coupled controller design is presented to make the multi-agent systems achieve consensus. At last, two numerical examples and a practical example are used to show the effectiveness of the results proposed in this paper. [ABSTRACT FROM AUTHOR]

The predefined-time cooperative output regulation of second-order nonlinear multiagent systems with an unknown exosystem is investigated in this paper. Existing similar work is almost limited to exactly known exosystems, and the upper bound of convergence time for closed-loop systems is difficult to adjust and predict, which may not meet practical needs. In this paper, we first propose a new dynamic compensator called predefined-time distributed observer, which guarantees that each agent estimates the state and parameters about the linear exosystem with unknown system matrix through the general directed communication network within the time set by an explicit parameter. Subsequently, combining the improved dynamic gain control method and the proposed predefined-time distributed observer, a novel distributed dynamic predefined-time control scheme is proposed, which solves the problem of this paper by simplifying the tedious arithmetic process caused by the traditional backstepping method. The final simulation experiments verify the effectiveness of the proposed control scheme. • We extended similar articles to more general leader and controlled objects. • The settling time of our controller can be set directly and structure is simple. • We present a new distributed observer for unknown leader and general digraph. • The designed observer can work for an arbitrarily set period of time. • We transform an unknown leader into a known one in a predefined time. [ABSTRACT FROM AUTHOR]

This paper studies the consensus problem of a new multi-agent system via multi-task, multi-control mechanism and multi-consensus strategy. Firstly, two multi-task algorithms are proposed, which can reasonably select some agent nodes with better performance to form the multi-agent system according to different tasks. Secondly, a multi-control protocol designed in this paper integrates continuous control mechanism, impulsive control mechanism and hybrid control mechanism, which has strong applicability and efficiency. Thirdly, a multi-consensus strategy is designed, and it can be converted into "average consensus strategy", "leader-following consensus strategy" and "hybrid consensus strategy" through different parameter settings, which greatly broadens the application field of the multi-agent system. We construct a dynamic model for the proposed multi-agent system, analyze it by using matrix measure and Lyapunov stability theory, and obtain the theorems that enable it to achieve multiple consensus. Finally, the efficiency and practicability of our theories are verified through several simulation experiments. We further carry out some comparative analysis with some relevant papers, and obtain the superiority of our theories in multiple indicators. [ABSTRACT FROM AUTHOR]

This paper focuses on the distributed online optimization problem in multi-agent systems considering privacy preservation. Each agent exchanges local information with neighboring agents on the strongly connected time-varying directed graphs. Since the process of information transmission is prone to information leakage, a distributed push-sum dual averaging algorithm based on the differential privacy mechanism is proposed to protect the privacy of the data. In addition, to handle situations where the gradient information of the node cost function is unknown, the one-point gradient estimation is designed to calculate the true gradient information and guide the update of the decision variables. With the appropriate choice of the stepsizes and the exploration parameters, the algorithm can effectively protect the privacy of agents while achieving sublinear regret with the convergence rate O (T 3 4 ). Furthermore, this paper also explores the effect of one-point estimation parameters on the regret in the online setting and investigates the relation between the convergence effect of individual regret and differential privacy levels. Finally, several federated learning experiments were conducted to verify the efficacy of the algorithm. [ABSTRACT FROM AUTHOR]

In this paper, for the multi-agent system, the Nash equilibrium (NE) search problem for non-cooperative game with state constraints under false data injection (FDI) attack is studied. The FDI attack can directly destroy the connectivity between agents, resulting in the performance degradation or even failure for most NE search algorithms. We propose a new distributed NE search method to improve the performance of NE algorithms under FDI attacks. The NE algorithm is designed to find the solution of NE under network attack and make the system recover after the attack. However, most of the existing results only studied the NE algorithm of the non-attacking system, and the existing results rarely take into account whether the system state exceeds the safety limit, which can cause the system to crash. This paper solves the state constraints problem of the multi-agent game under the FDI attack. Finally, the reliability of the algorithm is verified by a numerical result. [ABSTRACT FROM AUTHOR]

In this paper, the event-triggered output-based distributed observer (ETOBDO) is proposed for a leader system with linear dynamics over switching communication networks satisfying the jointly connected condition. Then ETOBDO is applied to solve the cooperative output regulation problem (CORP) of linear heterogeneous multi-agent systems (MASs) over jointly connected switching networks (JCSNs). Comparing with the similar results which only implement the distributed observer by the event-triggering mechanism, we implement both the distributed observer and the control law of each follower by the event-triggering mechanism. Thus, our overall output feedback control law can be directly implemented in a digital platform. Moreover, we give a rigorous proof for the prevention of Zeno phenomenon. A practical example adopted from the vehicle longitudinal dynamics is utilized to illustrate the effectiveness of the overall design. [ABSTRACT FROM AUTHOR]

In recent years, in the field of Multi-Agent Systems (MAS), significant progress has been made in the research of algorithms that combine Reinforcement Learning (RL) with Attention Mechanism (AM). However, there is a lack of comprehensive reviews in this field. Based on this, this paper does the following work. Firstly, it reviews the classical algorithms of RL and AM; Secondly, it systematically introduces the combination of RL and AM; Thirdly, it sorts out their application in the field of single-agent and multi-agent, and pays attention to and looks forward to the challenges and future research prospects in this field; The last part offers a comprehensive analysis of the challenges encountered by research in this field and anticipates future research paths. The research offers a conceptual understanding and theoretical foundation for future applications of RL using AM and facilitates further in-depth study in this field for researchers. [ABSTRACT FROM AUTHOR]

This paper considers the exponential consensus of nonlinear switched multi-agent systems (MASs) subject to random attacks. The proposed model is enhanced by introducing the concept of mode-dependent average dwell time (MDADT) and the one-sided Lipschitz (OSL) condition, making it more practical and versatile with relaxed restrictions. Due to the unavailability of direct access to the states of MASs and the vulnerability of their output signals to unknown random attacks, achieving consensus through traditional methods becomes challenging. Hence, a novel attack-decomposition approach is proposed to extract the attack-free output signals from the attacked output signals. Then, the observer is constructed by utilizing attack-free output signals and employed to estimate the states of MASs. A mode-dependent event-triggering mechanism is presented to conserve resources for the observer-controller channels. In order to save control costs, the intermittent control scheme is designed for different switching modes whose control width depends on the minimum dwell time. Finally, an illustrative example is used to showcase the advantages of the theoretical method. [ABSTRACT FROM AUTHOR]

In this paper, we discuss event-triggered consensus control for a heterogeneous second-order multi-agent systems with unknown nonlinear functions and state delays (SODMASs). Firstly, a broad adaptive dynamic event-triggered mechanism (ETM), which includes many existing ETM, is proposed. Based on this ETM and the approximation property of neural networks, an adaptive event-triggered protocol with weight estimation of neural network and time-varying integral gain compensation is designed to reduce communication frequency and to eliminate the uncertainty of nonlinear function and state delays. Then, the consensus problem for the heterogeneous SODMASs under the protocol is solved successfully by selecting a suitable Lyapunov–Krasovskii functional. In addition to these, it is also proved that all agents can exclude Zeno behavior under the proposed event-triggered protocol. Finally, a numerical example is given to verify the effectiveness of the proposed consensus protocol algorithm. [ABSTRACT FROM AUTHOR]

This paper concentrates on the bipartite finite-time consensus (BFTC) problem of nonlinear multi-agent systems (MASs) under intermittent communication via event-triggered impulsive control. First, a finite-time consensus (FTC) protocol with an event-triggered mechanism is developed for the MASs with nonlinear dynamics to save limited resources. Then, an impulsive control scheme is introduced to the designed algorithm to further improve the control performance. Based on the structurally balanced graph theory, both antagonistic and cooperative interactions are discussed in this article. Regarding whether the MASs have encountered intermittent communication, sufficient conditions are provided for achieving consensus of the MASs by using the Lyapunov theory and inductive methods. Moreover, the Zeno behavior is excluded. Finally, numerical simulations are given to validate the effectiveness of the algorithm. The result shows that the designed event-triggered BFTC control protocol with an impulsive strategy has a faster convergence rate than that without an impulsive strategy. [ABSTRACT FROM AUTHOR]

In this paper, distributed adaptive finite-time output feedback containment control based on command filters is proposed for nonstrict-feedback stochastic multi-agent systems (SMASs) with unmodeled dynamics, input quantization and prescribed performance. The unknown states are estimated using a high-gain observer, the unmodeled dynamics is handled by introducing a measurable dynamic signal, and the input signal is processed using an integrated quantizer that combines the advantages of uniform quantizer and hysteretic quantizer. The hyperbolic tangent function and time-varying function are applied to construct the performance function, and the constrained containment error is transformed into an unconstrained system through nonlinear mapping (NM). Unknown smooth functions are handled with the superb approximation capability of radial basis function neural networks (RBFNNs). Based on stochastic finite-time theory and dynamic surface control (DSC) method, nonlinear command filters are introduced to reduce the use of black-box functions and simplify the controller design. Finally, the theoretical analysis and two sets of experimental results demonstrate that all signals in the controlled system are semi-global finite-time stability in probability (SGFSP) and the designed controller works well. • Distributed adaptive finite-time output feedback containment control is developed for MASs. • The definition of achieving containment control in probability is clearly proposed. • A high-gain observer is designed to estimate unmeasurable states. • Novel Lyapunov function including the compensation errors is constructed. • The containment control and prescribed performance are fulfilled in probability. [ABSTRACT FROM AUTHOR]

In the past few decades, distributed multi-agent system (MAS) control has received growing attention due to its numerous advantages. Nonetheless, the substantial reliance on local information exchange in distributed MAS control has given rise to significant privacy concerns. Differential privacy (DP), a mathematically rigorous privacy notion, has gained popularity as a means of safeguarding privacy across multiple fields, including distributed MAS control. In this paper, we present an in-depth overview of the techniques for preserving DP in distributed MAS control, concentrating on consensus and distributed optimization. We begin by outlining the defining features and modeling of MASs from the control theory perspective. Then, we illustrate the motivation for adopting differentially private mechanisms to protect the privacy of distributed MAS control and present the fundamental principles of DP. Based on them, we investigate the cutting-edge techniques designed to preserve DP in consensus and distributed optimization. This review sheds light on the current landscape of DP applications in distributed MAS control and lays the groundwork for future progress in this essential field. [ABSTRACT FROM AUTHOR]

This paper primarily examines group consensus (GC) in heterogeneous multi-agent systems (HMASs) within the context of cooperative-competitive networks and input delay. The agent dynamics are modeled using single and double integrators, while the complex network encompassing these agents demonstrate both cooperative and competitive relationships. We first propose control protocols incorporating delays to address GC in HMASs. Secondly, we establish algebraic criteria employing frequency-domain analysis techniques to ensure the achievement of GC in directed network topologies for HMASs, which is equally applicable to undirected networks. Following this, we conduct an extensive discussion to determine the upper boundary for permissible input delays in HMASs. Finally, simulations are performed to validate our theoretical findings and the algorithms. [ABSTRACT FROM AUTHOR]

As an important branch of emerging artificial intelligence algorithms, multi-agent reinforcement learning (MARL) has shown strong performance in collaborative environments. It can utilize multiple agents to find the optimal set of strategies for solving sequential decision problem through trial-and-error. One of the main challenges facing multi-agent system is the non-stationarity problem, which brings poor convergence and seriously affects its performance. Clustering is a commonly used unsupervised analytical method in machine learning, which aims to group samples with similar internal properties into the same cluster. In this paper, we propose a MARL clustering algorithm based on silhouette coefficient (SC-MARLC), and use the trial-and-error strategy to find the best cluster groups. In SC-MARLC, we establish a mapping relationship between multi-agent and samples, construct a novel clustering model based on MARL, and design a good clustering subset structure based on the sample silhouette coefficient. The designed structure is helpful for multi-agent system to solve the non-stationary problem. Finally, we compare the performance of SC-MARLC with 11 existing clustering algorithms on fifteen public datasets. The results show that the new clustering algorithm performs best on ten datasets. [Display omitted] [ABSTRACT FROM AUTHOR]

In this paper, a distributed control algorithm based on a data-driven approach is developed to solve the consensus control of heterogeneous nonlinear Multi-Agent Systems (MAS). The consensus obtained from the solution of the Hamilton–Jacobi–Bellman (HJB) equation is challenging for unknown nonlinear systems. To address this issue, improved online reinforcement learning (RL) is employed to generate an approximate solution for each agent to achieve consensus. Unlike model-based RL and traditional algorithms, this method leverages I/O data to guide the learning of policies without any prior knowledge of agent dynamics. Furthermore, the adaptability of algorithm to heterogeneous nonlinear agents is enhanced by implementing online updates to the control strategy and dynamic linearization (DL). The convergence analysis of the algorithm is provided, along with the impact of the learning rate parameters on the consensus of MAS. By comparing with other data-driven methods, simulations are conducted to verify the stability and adaptability of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper studies the fixed-time containment of the second-order nonlinear multi-agent system (MAS) accompanied by switching topologies and event-triggered communication. The agent is subjected to unknown nonlinear dynamics. A fixed-time RBF observer was constructed based on information obtained from neural networks and event-triggered communication. By combining the controller with the identification information from the observer, it is demonstrated that the system can achieve practical fixed-time output feedback containment and exclude Zeno behavior during the control process. The effectiveness of the proposed event-triggered control method for fixed-time output feedback containment control of the second-order nonlinear MAS is verified by simulation examples. [ABSTRACT FROM AUTHOR]

Distributed optimization, combining cooperative control and optimization objectives, is vital for practical applications. While current research predominantly focuses on linear multi-agent systems, the real-world demands often involve nonlinear systems with finite or fixed time constraints. This paper addresses the finite-time/fixed-time distributed optimization for nonlinear multi-agent systems with time-varying cost function. Under appropriate assumptions, we propose a finite-time distributed protocol and a fixed-time distributed protocol for the nonlinear multi-agent systems to ensure all agents achieve consensus and minimize the time-varying cost function, relaxing the conditions for strong convexity and bounded gradient difference of the cost function. Simulation experiments validate the effectiveness of these distributed protocols. [ABSTRACT FROM AUTHOR]

The bumpless transfer consensus control problem for linear homogeneous multi-agent systems subject to agent-dependent switching directed communication topologies is investigated in this paper. According to the information of graph Laplacian matrix and relative state of agent, we propose a novel agent-dependent switching law to guide the switching among these topologies, then the leader-following consensus performance of multi-agent systems is achieved via using the multiple Lyapunov functions method. Further, since the switching consensus controllers are considered, the control signal bumps are inevitable at switching instants, and some sufficient conditions via linear matrix inequalities are developed to pursue the bumpless transfer performance of multi-agent systems. Besides, the agent-dependent switching law also allows the error dynamics under each directed subtopology to be unstabilizable. Finally, the theoretical results are verified by two illustrative examples. [ABSTRACT FROM AUTHOR]

This paper addresses the event-triggered global consensus issue of second-order multi-agent systems (MASs) with asymmetric input saturation, where the zero saturation bounds are allowed. First, for the leaderless situation, we propose an event-triggered mechanism and give the criterion for realizing global consensus. Different from the traditional event-triggered strategy in an effort to exclude the Zeno behavior, a preset minimum triggering interval is introduced. That is, the control input will not be updated within this interval after the latest trigger time, and the event-triggered condition is activated after this interval. Moreover, the above results are extended to the leader–follower case. Finally, some numerical examples illustrate the effectiveness of the developed schemes. [ABSTRACT FROM AUTHOR]

Consensus control in multi-agent systems (MASs) has wide-ranging applications in various domains. As the scope of applications continues to expand, ensuring the secure control of these systems to counter a diverse range of network attacks becomes highly necessary. Among these threats, Denial of Service (DoS) attacks have emerged as a major network security threat, capable of rendering targeted agents or networks unable to provide normal services. This paper presents an effective control protocol aiming to achieve leader-following consensus in a second-order MAS with event-triggered control mechanism and nonlinear dynamics under DoS attacks. The protocol utilizes absolute velocity and relative position measurements. The maximum attack frequency and duration of the DoS attacks have been obtained. Furthermore, the possibility of Zeno behavior is eliminated. Theoretical analysis demonstrates that, even under DoS attacks, consensus can be maintained among the leaders and followers within the MASs. In conclusion, the effectiveness of the proposed distributed security control protocol is validated through a case study involving a multi-vehicle cooperative system with unmanned cars. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the leader-following guaranteed performance consensus problem for multi-agent systems(MASs) controlled by event-triggered under external disturbances. First, a disturbance observer is designed for each follower, which can be used to efficiently estimate external disturbances. Second, to reduce the waste of communication resources, a new distributed event-triggered strategy based on the state estimation of neighboring agents is presented, which does not need continuous communication among agents. And the growing estimation error is reset by communication updates. The analysis illustrates that Zeno behavior can be effectively averted by formulating a lower bound on the number of normals for the minimum inter-event interval. Also, based on the above proposed state feedback consensus protocol with guaranteed performance, sufficient conditions are derived to guarantee that a MASs can achieve leader-following consensus. In addition, the consensus criterion with guaranteed performance is put forward in the form of a linear matrix inequality. The cost of guaranteed performance with leader-following structural properties is also established. At last, an example of numerical simulation is presented to confirm the derived analytical solution. [ABSTRACT FROM AUTHOR]

This paper investigates the flocking control issue of multi-agent systems, where each agent utilizes the broadcasting communication manner and takes the Cucker–Smale style dynamic model. In the setting of broadcasting scenario, each agent broadcasts the state information only at certain moments determined by its own clock, and recalculates the control input only when its clock ticks or it receives information from the neighbors. In the Cucker–Smale model, a nonlinear weight function about the communication distance is set to portray the mutual influence degree between agents. With the help of the properties of sub-stochastic matrix, a comprehensive analysis of the asymptotic convergence of the model is performed and the algebraic conditions for achieving flocking control are established. In addition, the obtained theoretical results are further extended to the bipartite flocking control issue for multi-agent systems with both cooperative and competitive interactions. At last, numerical simulations are provided to verify the theoretical results. [ABSTRACT FROM AUTHOR]

This paper investigates the enclosing control of hybrid multi-agent systems under directed networks. Firstly, we establish some criteria for continuous-time and discrete-time multi-agent systems to achieve enclosing control. Secondly, according to the communication modes among agents, four distributed enclosing control protocols are proposed for hybrid multi-agent systems. Then, under the proposed protocols, we give the corresponding sufficient conditions to guarantee enclosing control. Finally, the correctness of our results is verified by simulations. [ABSTRACT FROM AUTHOR]

In this paper, a distributed economic dispatch algorithm based on event-triggered and fixed-time consensus is proposed for optimal dispatching among generators in smart grid. Aiming to quickly achieve economic dispatch in distributed smart grid and significantly reduce the frequency of communication between generators, the algorithm can address the supply–demand balance between the generation side and the consumption side, as well as the capacity constraints of the generators. Different from the relevant work, the proposed incremental cost can converge in fixed-time and the settling time is independent of the initial state of the generators. It will provide more flexibility for the system. Meanwhile, a fully distributed event-triggered strategy is adopted, which can reduce the interactive information between generators and avoid the waste of communication resources. Furthermore, the proposed algorithm is robust to the time-varying communication network topology. Finally, several examples are presented that validate the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

A well-known drawback in distributed average consensus of multi-agent systems is that the exchanged information is usually delayed due to the time elapsed during the data transmission process. Using classical dynamic average consensus, delays may lead to poor performance or even instability. In this paper, we propose a novel dynamic consensus method that counteracts the negative effects of delays by means of delay compensation techniques. The interest of our dynamic consensus method with delay compensation is that it converges under mild conditions on graph connectivity and bounded reference signals, no matter how large the delays are, as long as delays are fixed and known. We also provide a formal characterization of the convergence speed of our method. Additionally, our results apply to fixed directed strongly connected, and undirected topologies. • We propose a dynamic consensus method for discrete-time systems. • We include a predictor feedback strategy for delay compensation. • Our results apply to fixed directed graphs that are strongly connected. • These results rely on a thorough eigenvalue-based analysis. • We propose parameter tuning rules aimed at improving the speed of convergence. [ABSTRACT FROM AUTHOR]

This paper investigates the fully distributed consensus control problem for general linear multi-agent systems via dynamic event-triggered scheme. To address the need to reduce network communication burdens and avoid involving any global features of multi-agent networks, a novel fully distributed controller based on the open-loop state estimator of agents is presented. To improve anti-interference ability and practicability of the proposed approach, an event-triggered strategy which can guarantee a precise lower bound of triggering intervals for each follower is designed with a virtual internal variable. Then, the stability of the multi-agent systems is analyzed without any global network information. Several simulation experiments are conducted to illustrate theoretical results. [ABSTRACT FROM AUTHOR]

This paper explores the problem of distributed cooperative tracking control for networked nonlinear multi-agent systems (MASs) with communication delays. The difficulty of distributed control of subsystems across several geographic areas is addressed by utilizing MASs in this study. To overcome modeling difficulties, a model-free control algorithm is employed. To handle networked delay, a network predictive control approach is utilized. To address these issues in a unified manner, a novel distributed model-free integral sliding mode predictive control strategy is proposed. This approach requires simple data to make predictions to achieve distributed tracking performance while also actively compensating for delays. The effectiveness of the proposed method is demonstrated through simulations and experimental results. [ABSTRACT FROM AUTHOR]

This paper proposes optimal bipartite consensus issue for discrete-time multi-agent system (DTMAS) using event-triggered control (ETC) and quantized event-triggered control (QETC). Firstly, a novel event-triggered control strategy is introduced along with specific conditions to ensure the stability for DTMAS. Due to the intractability of the Hamilton–Jacobi–Bellman (HJB) equation, through two neural networks (NNs), the value function and optimal control are approximated by the adaptive dynamic programming (ADP) method. The weight matrix of the action neural network is updated only when triggered, while also taking into account the quantization effect of the information. The stability of the weights errors and the convergence of the system dynamic are demonstrated using Lyapunov stability theory. Finally, the practicality of the improved method is confirmed a numerical example. [ABSTRACT FROM AUTHOR]