Search

In nowadays production, fluctuations in demand, shortening product life-cycles, and highly configurable products require an adaptive and robust control approach to maintain competitiveness. This approach must not only optimise desired production objectives but also cope with unforeseen machine failures, rush orders, and changes in short-term demand. Previous control approaches were often implemented using a single operations layer and a standalone deep learning approach, which may not adequately address the complex organisational demands of modern manufacturing systems. To address this challenge, we propose a hyper-heuristics control model within a semi-heterarchical production system, in which multiple manufacturing and distribution agents are spread across pre-defined modules. The agents employ a deep reinforcement learning algorithm to learn a policy for selecting low-level heuristics in a situation-specific manner, thereby leveraging system performance and adaptability. We tested our approach in simulation and transferred it to a hybrid production environment. By that, we were able to demonstrate its multi-objective optimisation capabilities compared to conventional approaches in terms of mean throughput time, tardiness, and processing of prioritised orders in a multi-layered production system. The modular design is promising in reducing the overall system complexity and facilitates a quick and seamless integration into other scenarios. [ABSTRACT FROM AUTHOR]

Trading decisions often encounter risk and uncertainty complexities, significantly influencing their overall performance. Recognizing the intricacies of this challenge, computational models within information systems have become essential to support and augment trading decisions. The paper introduces the concepts of trading software agents, investment strategies, and evaluation functions that automate the selection of the most suitable strategy in near real-time, offering the potential to enhance trading effectiveness. This approach holds the promise of significantly increasing the effectiveness of investments. The research also seeks to discern how changing market conditions influence the performance of these strategies, emphasizing that no single agent or strategy universally outperforms the rest. In summary, the overarching objective of this research is to contribute to the realm of financial decision-making by introducing a pragmatic platform and strategies tailored for traders, investors, and market participants in the FOREX market. Ultimately, this endeavor aims to empower people with more informed and productive trading decisions. The contributions of this work extend beyond the theoretical realm, demonstrating a commitment to address the practical challenges faced by traders and investors in real-time decision-making within the financial markets. This multidimensional approach to financial decision support promises to enhance investment effectiveness and contribute to the broader field of algorithmic trading. [ABSTRACT FROM AUTHOR]

An increasing number of exchanges, mainly in the U.S., have adopted a commission structure called maker–taker fees in which traders placing limit orders (makers) are paid a rebate (negative trading commission) and traders placing market orders (takers) are charged a trading fee. The reason is that by paying rebates to makers, exchanges can expect to receive a large number of maker's orders and gain market share. Makers include arbitrageurs who make large transactions. Maker–taker fees constitute one of the most important commission structures for exchanges, because they are expected to attract arbitrageurs who are looking for rebate profits, on top of their trading profits. There have been many studies about arbitrage trading, but none we could find focused on the impact of arbitrage trading between markets with maker–taker fees where arbitrage traders place limit orders and markets without maker–taker fees where they place market orders. In this study, we investigated volatility and market liquidity by changing the amount of rebate under our proposed artificial markets, where there are or are not maker–taker fees. Then we checked the performance of arbitrage trading when the rebate increased. The results were that volatility in the market with maker–taker fees decreased and that in the market without maker–taker fees increased, and that market liquidity and arbitrage performance both increased in the market with maker–taker fees when rebates increased. The above results indicate that exchanges that operate markets adopting maker–taker fees can provide investors with more attractive markets than those that do not adopt them. However, if more arbitrageurs participate in the market with maker–taker fees to take advantage of these rebates, the cost burden on exchanges may increase unnecessarily. [ABSTRACT FROM AUTHOR]

Autonomous maneuvering decision-making is a crucial technology for Unmanned Aerial Vehicles (UAVs) to take the air domination in modern unmanned warfare. With the advantage of balancing exploration and exploitation, as well as the immediacy of end-to-end output by combining with deep neural network, multi-agent reinforcement learning (MARL) has made remarkable achievements in multi-UAV autonomous air combat maneuvering decision-making (MUAAMD). However, the implementation of effective cooperative policy learning remains a challenging issue for MARL methods with centralized training decentralized execution (CTDE) paradigm. This paper proposes a MARL-based method to improve the performance of cooperation in MUAAMD. Firstly, considering the constraints of dynamic and limited perception for UAVs in the realistic air combat scenario, the MUAAMD problem is formulated based on partially observable Markov game (POMG) model. Secondly, a novel efficient MARL algorithm named the mean policy-based proximal policy optimization (MP3O) is introduced. Specifically, a joint policy optimization mechanism is constructed by estimating the policies of neighboring agents in group as a mean-field approximation while training, which enables both centralized evaluation and improvement of cooperative policy under the CTDE paradigm. Thirdly, by combining with three improvement techniques, a cooperative decision-making framework for MUAAMD based on MP3O is proposed. Empirically, results of simulations and comparative experiments validate the effectiveness of proposed method in promoting cooperative policy learning in resolving MUAAMD problem. [ABSTRACT FROM AUTHOR]

Global Nash equilibrium is an optimal solution for each player in a graphical game. This paper proposes an iterative adaptive dynamic programming-based algorithm to solve the global Nash equilibrium solution for optimal containment control problem with robustness analysis to the iterative error. The containment control problem is transferred into the graphical game formulation. Sufficient conditions are given to decouple the Hamilton–Jacobi equations, which guarantee the solvability of the global Nash equilibrium solution. The iterative algorithm is designed to obtain the solution without any knowledge of system dynamics. Conditions of iterative error for global stability are given with rigorous proof. Compared with existing works, the design procedures of control gain and coupling strength are separated, which avoids trivial cases in the design procedure. The robustness analysis exactly quantifies the effect of the iterative error caused by various sources in engineering practice. The theoretical results are validated by two numerical examples with marginally stable and unstable dynamics of the leader. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of robust coordination control based on interval observers for multi‐agent systems in the presence of input saturation and disturbances. Firstly, in the scenario where the system state is not directly measurable, two types of interval observers are, respectively, constructed by resorting to the upper and lower bounds on external perturbances as well as the output information. Secondly, a parametric Lyapunov equation‐based low‐gain feedback control method is proposed to guarantee semiglobal bounded consensus. In contrast to the conventional approach based on parametric algebraic Riccati equations, the proposed method offers the advantage of allowing the parameters to be determined in advance. Finally, a simulation example and a practical electrical circuit model are conducted to verify the theoretical results. [ABSTRACT FROM AUTHOR]

Process design improvements require extensive knowledge, considerable time, and huge human resources due to the complexity of chemical processes and their diverse objective functions. However, machine learning-based approaches using vast accumulated data are limited in low versatility, applicable only to specific processes, and unable to understand the basis of model decisions. This study proposes the GPT-based Improved Process Hybrid Transformer (GIPHT), a process design improvement generation system utilizing Large Language Model (LLM). LLMs, being natural language-based, allow for understanding the basis of model decisions without need of explainable AI analysis. GIPHT is composed of multi-agent to enhance versatility and performance for diverse chemical processes. We also propose the Detailed Simplified Flowsheet Input Line Entry System format to express process diagrams in natural language, including enhanced information about process conditions. A structured prompt system is employed and validated in the LLM domain through prompt engineering. GIPHT searches and extracts data based on its proposed improvement methodology, providing explanations for the decision-making process and the basis, overcoming limitations of the traditional black-box AI models. It offers directional ideas to design engineers in the early stages of process design and would be used for training of process engineers, supporting improvement of outdated processes and transformation into more environmentally friendly processes. [ABSTRACT FROM AUTHOR]

As the power industry becomes increasingly complex, Digital Twin (DT) technology has emerged as a crucial tool for enhancing grid resilience and operational efficiency by creating dynamic digital replicas of physical systems. These replicas enable accurate simulations and proactive management, but the vast amount of data generated by DT systems poses significant challenges for processing and analysis. Cloud computing offers a flexible solution by offloading these computational tasks to distributed resources, allowing for real-time analysis and scalable operations. However, this approach introduces complexities in task distribution and maintaining quality of service. Recent efforts have applied Deep Reinforcement Learning (DRL) to address these challenges, primarily using single-agent methods. However, these methods struggle with scalability and performance in increasingly complex cloud environments. To overcome these limitations, we propose an efficient task scheduling framework based on Multi-Agent Deep Q-Network (MADQN) principles, specifically designed to optimize both response times and operational costs. We provide a comprehensive design overview of our approach and conduct a thorough evaluation of its performance. The experimental results clearly indicate that our approach can considerably reduce response times amd lower operational costs compared to current methods, including state-of-the-art single-agent approaches. [ABSTRACT FROM AUTHOR]

Minimum circle circumnavigation is proposed in this paper, which is of special value in target monitoring, capturing and/or attacking. With this new pattern, our goal is to steer one agent to enclose multiple targets along a safe and minimum circular orbit with the desired tangential speed. This is achieved by a control protocol with dynamic compensators, which is also capable of avoiding collision between the agent and the multiple targets during the whole operational process. The control protocol is further extended to the scenarios of multiple agents circulating a group of targets by adding a coordination mechanism into the tangential velocity term, which allows the agents to distribute evenly on each expected circular orbit with the same or different radius. Simulations results illustrate the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Finite-time consensus problem of hybrid-order heterogeneous multi-agent systems under a signed digraph topology is investigated in this paper. For heterogeneous multi-agent systems composed of first-order and second-order agents, a novel pinning event-triggered control protocol is devised to facilitate the attainment of the desired consensus state within a finite time. This control method overcomes communication barriers between first-order and second-order multi-agent systems, achieving effective control performance while reducing controller update frequency and communication costs. Based on graph theory and the Lyapunov stability method, several novel matrices are defined to address the finite-time consensus problem in hybrid-order multi-agent systems, and these matrices also facilitate the theoretical derivation process. Furthermore, it is demonstrated that the control protocol designed for hybrid-order systems is devoid of Zeno behavior. Finally, a detailed numerical example is supplied to illustrate the validity of the theoretical analysis. [ABSTRACT FROM AUTHOR]

This paper proposes a flexible path-planning method for conducting agricultural observation tasks using multiple autonomous guided vehicles. The observation task was formulated as the rural postman problem by graphing the agricultural field to be observed and assigning costs to mandatory and optional paths. The observation area was re-divided according to the progress of each robot's work, and the latest path was planned to minimize the difference between the working time of individual robots, thereby reducing the overall working time. Simulations were conducted to verify the influence of the parameters used in the proposed method. The effect of workload adjustment on progress delay was verified, and the increase in overall working time owing to delay was reduced by an average of 32.9%. A performance comparison was conducted using Google OR-Tools, software specialized for combinatorial optimization. Superior solutions with shorter computation times were obtained using the proposed method. [ABSTRACT FROM AUTHOR]

In this study, we propose a model called LFSPN, which serves as an extension of stochastic Petri nets dedicated to the multi-agent systems paradigm. The main objective is to specify, verify, validate, and evaluate the flow of materials within an automated production chain. We illustrate the practicality of our model by engaging in a systematic process of modeling and simulation of a production chain involving material flow. To evaluate the performance, we employ a mobile learning agent, which has distinct characteristics, namely mobility and learning. So, the distinctive characteristics of the learning agent are manifested in two key behaviors: mobility and learning. Notably, the learning agent is equipped with a flexible learning algorithm that integrates stochastic elements based on transitions. We suggest using a MATLAB simulation to determine the firing time of each transition within a sequence, guided by three different probability laws (exponential, normal, and log-normal). This sequence is designed to optimize the production process objective while facilitating learning cycles through agent rewards, specified by a production and consumption of tokens in our evolving model. We validate the effectiveness of our model by performing a comparative analysis with similar existing works. The advantages of our LFSPN model are twofold. Firstly, it offers a representation with two levels of abstraction: a graph representing the classic components of an SPN, and an additional layer encompassing the learning and migration aspects inherent to a mobile learning agent. Secondly, our model stands out for its flexibility and simulation simplicity. [ABSTRACT FROM AUTHOR]

Collaboration among road agents, such as connected autonomous vehicles and roadside units, enhances driving performance by enabling the exchange of valuable information. However, existing collaboration methods predominantly focus on perception tasks and rely on single-frame static information sharing, which limits the effective exchange of temporal data and hinders broader applications of collaboration. To address this challenge, we propose CoPnP, a novel collaborative joint perception and prediction system, whose core innovation is to realize multi-frame spatial–temporal information sharing. To achieve effective and communication-efficient information sharing, two novel designs are proposed: (1) a task-oriented spatial–temporal information-refinement model, which filters redundant and noisy multi-frame features into concise representations; (2) a spatial–temporal importance-aware feature-fusion model, which comprehensively fuses features from various agents. The proposed CoPnP expands the benefits of collaboration among road agents to the joint perception and prediction task. The experimental results demonstrate that CoPnP outperforms existing state-of-the-art collaboration methods, achieving a significant performance-communication trade-off and yielding up to 11.51%/10.34% Intersection over union and 12.31%/10.96% video panoptic quality gains over single-agent PnP on the OPV2V/V2XSet datasets. [ABSTRACT FROM AUTHOR]

It is challenging to quickly restore full connectivity in a damaged multi-agent self-organizing network while maintaining the residual connectivity structure. Therefore, this paper proposed a connectivity restoring method based on subnet fusion for self-organized networks in multi-agent systems. Firstly, the method designed a subnet partition algorithm based on network fault detection, to identify faulty nodes and subnet fragmentation in the system. Secondly, the method deployed a leaderfollower mobility model within each subnet to maintain the residual network connectivity. Finally, the method designed a reinforcement learning-based subnet fusion algorithm for leader election, where elected leaders periodically according to a reward function related to mobility distance and energy consumption, being responsible for guiding their followers to move for fusion between subnets. The experimental results show that this method reduces average restoration time by 11.3% and decreases energy consumption by 10.58%, demonstrating its advantages in efficiency and energy usage. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Beijing University of Technology is the property of Journal of Beijing University of Technology, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Large-scale indoor 3D reconstruction with multiple robots faces challenges in core enabling technologies. This work contributes to a framework addressing localization, coordination, and vision processing for multi-agent reconstruction. A system architecture fusing visible light positioning, multi-agent path finding via reinforcement learning, and 360° camera techniques for 3D reconstruction is proposed. Our visible light positioning algorithm leverages existing lighting for centimeter-level localization without additional infrastructure. Meanwhile, a decentralized reinforcement learning approach is developed to solve the multi-agent path finding problem, with communications among agents optimized. Our 3D reconstruction pipeline utilizes equirectangular projection from 360° cameras to facilitate depth-independent reconstruction from posed monocular images using neural networks. Experimental validation demonstrates centimeter-level indoor navigation and 3D scene reconstruction capabilities of our framework. The challenges and limitations stemming from the above enabling technologies are discussed at the end of each corresponding section. In summary, this research advances fundamental techniques for multi-robot indoor 3D modeling, contributing to automated, data-driven applications through coordinated robot navigation, perception, and modeling. [ABSTRACT FROM AUTHOR]

This paper introduces an effective intrusion detection system (IDS) for the internet of things (IoT) that employs a conflict-driven learning model within a multi-agent architecture to enhance network security. A double deep Q-network (DDQN) reinforcement learning algorithm is implemented in the proposed IDS with two specialized agents, the defender and the challenger. These agents engaged in an antagonistic adaptation process that dynamically refined their strategies through continual interaction within a custom-made environment designed using OpenAI Gym. The defender agent aims to identify and mitigate threats by matching the actions of the challenger agent, which is designed to simulate potential attacks in the environment. The study introduces a binary reward mechanism to encourage both agents to explore and exploit different actions and discover new strategies as a response to adversarial actions. The results showcase the effectiveness of the proposed IDS in terms of higher detection rate the comparative analysis also validates the effectiveness of the proposed IDS scheme with an accuracy of approximately 96%, outperforming similar existing approaches. [ABSTRACT FROM AUTHOR]

Economic models based on multi-agents are increasingly attracting attention and can provide a new perspective for exploring the causes behind social phenomena at the individual level. Existing research usually adopts society-level learning methods, and more research on micro-level heterogeneity among individuals is needed. For this, we propose a high-fidelity multi-agent economy (HMAE) model based on evolutionary game theory, including three types of agents: workers, firms, and the government. In particular, we characterize worker heterogeneity regarding laziness factors, work endowments, and commuting distances. These agents continuously and iteratively update their strategies by randomly exploring and imitating their neighbors to maximize their utility value. We simulated the evolution process of agent behavioral decisions through experiments and found that individual heterogeneity can significantly affect the decisions of workers and firms. These phenomena are consistent with some economic evolution trends in real life, and our research can provide an analytical tool for analyzing the causes of emerging economic phenomena. [ABSTRACT FROM AUTHOR]

To measure the effectiveness of learning activities, intensive research works have focused on the process of competency building through the identification of learning stages as well as the setup of related key performance indictors to measure the attainment of specific learning objectives. To organize the learning activities as per the background and skills of each learner, individual learning styles have been identified and measured by several researchers. Despite their importance in personalizing the learning activities, these styles are difficult to implement for large groups of learners. They have also been rarely correlated with each specific learning stage. New approaches are, therefore, needed to intelligently coordinate all the learning activities while self-adapting to the ongoing progress of learning as well as to the specific requirements and backgrounds of learners. To address these issues, we propose in this paper a new framework for an intelligent management of the competency building process during learning. Our framework is based on a recursive spiral Assess-Predict-Oversee-Transit model that is orchestrated by a multi-agent system. This system is particularly responsible of enabling smart transitions between learning stages. It is also responsible of assessing and predicting the process of competency building of the learner and, then, making the right decisions about the learning progress, accordingly. Results of our solution were demonstrated via an Augmented Reality app that we created using the Unity3D engine to train learners on Air Conditioner maintenance. [ABSTRACT FROM AUTHOR]

Most of the previous SOC equalization methods for microgrid energy storage target DC microgrids and use centralized control structures, while in recent years many researchers have begun to focus on a decentralized, communication-based implementation of distributed control structures. In this paper, based on the existing research, we use the multi-agent system (MAS) structure and dynamic consistency algorithm (DCA) to realize the estimation of the SOC mean value of microgrid energy storage obtained by each agent in the system. For the problems of the iterative convergence that exists in the distributed control, such as being too slow, too frequent communication, stability, etc., we optimally select the parameters of the consistency iterative summation to improve convergence. In addition, we use the event-driven method to further reduce the unnecessary communication frequency. Finally, a numerical simulation model of the AC microgrid is established by Matlab to verify the effectiveness of the method, which reduces the communication volume by about 50% while maintaining the effect of the control strategy. [ABSTRACT FROM AUTHOR]

The distributed estimation technology is prevalently utilized to solve the leader‐following multi‐agent tracking problem. This technology poses a challenge in practice, since it generally relies on the available absolute state measurements. For this reason, a novel distributed estimation approach based on relative state measurements is developed in this article. The proposed method directly estimates the tracking error between the leader and each follower, rather than using an existing indirect way of estimating and making subtraction under absolute state measurements. Specifically, a distributed directed estimation is first studied to complete estimation tasks within prescribed time under the known directed networks. Then, a fully distributed directed estimation problem is considered under the unknown directed networks. Both distributed and fully distributed results are extended to the robustness cases to resist external disturbances. Simulation examples, including numerical examples and a multiship coordination example, are provided to demonstrate the effectiveness and advantages of the proposed distributed estimation method. [ABSTRACT FROM AUTHOR]

The formation obstacle avoidance control problem of multi-agent systems was studied for multiple anonymous agents in the two-dimensional plane. The goal is to make the anonymous agents randomly distributed in the plane complete the obstacle avoidance progressive convergence to the target circle within the specified time, and then complete the specified formation on the circle according to the formation control law. Based on the artificial potential field method, the function of avoiding obstacles was realized, and the problem that the agent cannot move in the local minimum area was solved by adding random disturbance. The Lyapunov function was introduced and the stability of the control law designed was proved by theorem. Finally, the effectiveness of the obstacle avoidance function and the accuracy of the control law were proved by simulation. [ABSTRACT FROM AUTHOR]

Plug & Produce aims to revolutionize manufacturing by enabling seamless machine integration into production processes without extensive programming. This concept, leveraging multi-agent systems (MAS), offers increased flexibility and faster production ramp-up times after reconfiguration. As automated manufacturing moves towards greater human integration, this paper addresses safe operation within the Plug & Produce concept. The main safety challenge arises from autonomous decision-making, as agents in the MAS lack awareness of the risk consequences of their behavior. Additionally, the difficulty of perceiving the system's exact behavior leads to the implementation of overly restrictive safety measures. This limits the system's flexibility and ability to make decisions for efficient production. This paper proposes a method utilizing multi-agent control to conduct automatic safety analysis and reason task allocations to avoid risks. The method's benefits are the generation of control actions that comply with safety requirements during operation, eliminating the need for overly restrictive safety measures and allowing more effective equipment utilization. The method's benefit is illustrated through a manufacturing scenario with two different configurations: one using a hazardous machine and the other using a less hazardous one. Formal verification using the model checker NuSMV demonstrated that safety requirements were satisfied in both configurations, without the need for manual modifications of the safety control system after reconfiguration. The results for this specific manufacturing scenario showed that there are more reachable states (20 states) in the safer second configuration, compared to the first configuration (16 states). This means that the presented control strategy dynamically adjusts the system's behavior to confirm safety. Hence, this method maintains safety without fixed safety rules that limit the operations. [ABSTRACT FROM AUTHOR]

The need for self‐adaptive and intelligent supply chain systems is essential to meet the challenges of the current global markets. Despite the recent breakthroughs in artificial intelligence, literature still lacks the application of state‐of‐the‐art methods to optimize the performance of supply chain ordering management problems. Thus, this paper proposes a relatively new Deep Reinforcement Learning‐based Ordering Mechanism (DRLOM) for multi‐echelon linear supply chain systems. Initially, the supply chain ordering management problem is formulated as an agent‐based reinforcement learning model and, afterwards, solved using a recently developed policy‐based algorithm called proximal policy optimization. The proposed approach (DRLOM) aids the assumed supply chain echelons, such as the Retailer, the Wholesaler, the Distributor and the Factory, to learn the optimal/near‐optimal dynamic strategies for inventory ordering systems. The experimental results also validate that the proposed approach efficiently minimizes the system‐wide total accumulated inventory costs under different problem instances than other ordering heuristics and evolutionary computation methods. Throughout this paper, benchmark findings from the literature are used to evaluate the performance of the proposed approach. Furthermore, limitations of the earlier works are addressed through this paper and contribute to the supply chain ordering management literature. [ABSTRACT FROM AUTHOR]

This paper addresses and acknowledges the valuable feedback provided by Dr. Deniz Preil in response to the recent study conducted by Kurian et al which investigates the application of proximal policy optimization (PPO) to determine dynamic ordering policies within multi‐echelon supply chains. The first comment raised by Dr. Preil motivated an examination of the training and evaluation procedures in Experiments 2, 3, and 4. The Experiments 2 and 3 were reworked to address this, allowing the seed to vary for every training iteration, resulting in refined outcomes while there was no need of reworking of Experiment 4. The second comment focused on the benchmarking strategies involving the 1‐1 policy and the order‐up‐to (OUT) policy, clarifying the distinctions between the two policies and justifying the use of the 1‐1 policy for benchmarking in Experiment 4. The implementation of the widely accepted OUT policy was explained, highlighting the meaningful rationale behind its use. These discussions aim to enhance the methodology employed by Kurian et al and strengthen the implications of the findings within the domain of supply chain ordering management. [ABSTRACT FROM AUTHOR]

Aiming at the problems of low solution accuracy and high decision pressure when facing large-scale dynamic task allocation (DTA) and high-dimensional decision space with single agent, this paper combines the deep reinforcement learning (DRL) theory and an improved Multi-Agent Deep Deterministic Policy Gradient (MADDPG-D2) algorithm with a dual experience replay pool and a dual noise based on multi-agent architecture is proposed to improve the efficiency of DTA. The algorithm is based on the traditional Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, and considers the introduction of a double noise mechanism to increase the action exploration space in the early stage of the algorithm, and the introduction of a double experience pool to improve the data utilization rate; at the same time, in order to accelerate the training speed and efficiency of the agents, and to solve the cold-start problem of the training, the a priori knowledge technology is applied to the training of the algorithm. Finally, the MADDPG-D2 algorithm is compared and analyzed based on the digital battlefield of ground and air confrontation. The experimental results show that the agents trained by the MADDPG-D2 algorithm have higher win rates and average rewards, can utilize the resources more reasonably, and better solve the problem of the traditional single agent algorithms facing the difficulty of solving the problem in the high-dimensional decision space. The MADDPG-D2 algorithm based on multi-agent architecture proposed in this paper has certain superiority and rationality in DTA. [ABSTRACT FROM AUTHOR]

Simulation technology has been extensively utilized in the study of ship entry and exit from ports, as well as navigation through waterways. It effectively mirrors the stochastic dynamic changes and interrelationships among various elements within the port system. This paper provides a comparative analysis of the advantages and disadvantages of various modeling methods used in ship navigation simulations. It proposes a simulation modeling approach for ship–port systems based on multi-agent information interaction, which simulates the entire process of ships entering and exiting ports and navigating through complex waterways, achieving a precise and detailed simulation of the entire port entry and exit process in complex waters. Using the Qiongzhou Strait as a case study, the validity and accuracy of the model are verified. The model is employed to quantitatively identify port navigation elements, assess waterway capacity, and evaluate port operational capability. Furthermore, the model enables the analysis of coordination among port channels, berths, and anchorages. Based on simulation results and port development plans, recommendations are provided to enhance port service levels and promote scientific, rational development and efficient operation of ports. [ABSTRACT FROM AUTHOR]

Transhumants move their herds based on strategies simultaneously considering several environmental and socio-economic factors. There is no agreement on the influence of each factor in these strategies. In addition, there is a discussion about the social aspect of transhumance and how to manage pastoral space. In this context, agent-based modeling can analyze herd movements according to the strategy based on factors favored by the transhumant. This article presents a reductionist agent-based model that simulates herd movements based on a single factor. Model simulations based on algorithms to formalize the behavioral dynamics of transhumants through their strategies. The model results establish that vegetation, water outlets and the socio-economic network of transhumants have a significant temporal impact on transhumance. Water outlets and the socio-economic network have a significant spatial impact. The significant impact of the socio-economic factor demonstrates the social dimension of Sahelian transhumance. Veterinarians and markets have an insignificant spatio-temporal impact. To manage pastoral space, water outlets should be at least 15 k m from each other. The construction of veterinary centers, markets and the securitization of transhumance should be carried out close to villages and rangelands. [ABSTRACT FROM AUTHOR]

We consider the controllability of multi-agent dynamical systems modeled by a particular class of bipartite graphs, called chain graphs. Our main focus is related to chain graphs with exactly one repeated degree. We determine all chain graphs with this structural property and derive some properties of their Laplacian eigenvalues and associated eigenvectors. On the basis of the obtained theoretical results, we compute the minimum number of leading agents that make the system in question controllable and locate the leaders in the corresponding graph. Additionaly, we prove that a chain graph with exactly one repeated degree, that is not a star or a regular complete bipartite graph, has the second smallest Laplacian eigenvalue (also known as the algebraic connectivity) in (0.8299,1) and we show that the second smallest eigenvalue increases when the number of vertices increases. This result is of a particular interest in control theory, since families of controllable graphs whose algebraic connectivity is bounded from below model the systems with a small risk of power or communication failures. [ABSTRACT FROM AUTHOR]

Time-delay is an unavoidable factor in analyzing and designing any networked system since in most scenarios, it decreases the performance or even destabilizes the system. In this paper, we study the newly proposed matrix-scaled consensus (MSC) algorithms under the presence of communication time delays. Both networks of single- and double-integrator agents are considered, and for each scenario, a corresponding delay margin will be given. The analysis hinges on the Nyquist stability criteria and the algebraic solution of quasi-polynomials associated with the MSC models. Numerical examples are provided to demonstrate the correctness of theoretical results. [ABSTRACT FROM AUTHOR]

Abstract As the power industry becomes increasingly complex, Digital Twin (DT) technology has emerged as a crucial tool for enhancing grid resilience and operational efficiency by creating dynamic digital replicas of physical systems. These replicas enable accurate simulations and proactive management, but the vast amount of data generated by DT systems poses significant challenges for processing and analysis. Cloud computing offers a flexible solution by offloading these computational tasks to distributed resources, allowing for real-time analysis and scalable operations. However, this approach introduces complexities in task distribution and maintaining quality of service. Recent efforts have applied Deep Reinforcement Learning (DRL) to address these challenges, primarily using single-agent methods. However, these methods struggle with scalability and performance in increasingly complex cloud environments. To overcome these limitations, we propose an efficient task scheduling framework based on Multi-Agent Deep Q-Network (MADQN) principles, specifically designed to optimize both response times and operational costs. We provide a comprehensive design overview of our approach and conduct a thorough evaluation of its performance. The experimental results clearly indicate that our approach can considerably reduce response times amd lower operational costs compared to current methods, including state-of-the-art single-agent approaches.

One of the directions of development of the electric power industry is decentralization, aimed at improving the reliability of energy supply, reducing losses during transmission of electric energy and ensuring energy independence of consumers. It is possible to simulate decentralized power systems, including distributed generation facilities, by implementation of multi-agent systems that allow solving design and control problems taking into account the needs of each participant in the process of production, transmission, distribution and consumption of electricity. The development of distributed generation using a multi-agent approach requires the creation of models for assessing the technical and economic efficiency of decisions made by each agent, both at the strategic and tactical levels. The strategic decisions of agents related to distributed generation include, among other things, the creation of power facilities and power plants based on renewable energy sources. An important factor for making such decisions is the estimation of the capacity factor. However, currently there are no models for its estimation with high reliability. The present paper proposes a new algorithm for estimating the capacity factor for the entire territory of a certain administrative unit and a model for its forecasting based on climatic and geographical parameters. The study was conducted on a data sample of 221 generation facilities (solar and wind power plants) in four oblasts (regions) of the Russian Federation. It has been determined that the capacity factor can be forecasted with a mean error within 4 % for photovoltaic power plants and 9 % for wind power plants. Therefore, it is possible to use the developed algorithm and model both in decision support systems when choosing the location of this types of power plants, and in systems that model the development of power systems using a multi-agent approach.

Abstract Distributed control can increase system scalability, flexibility, and redundancy. Foundational to such scalability via decentralisation is consensus formation, by which decision-making and coordination are achieved. However, decentralised multi-agent systems are inherently vulnerable to disruption. To develop a resilient consensus approach, inspiration is taken from the study of social dynamics; specifically, the Deffuant Model which evaluates the impact of tolerance in social systems. A dynamic protocol is presented enabling efficient consensus to be reached with an unknown number of disruptors present within a multi-agent system. By inverting typical social tolerance, agents filter out extremist non-standard opinions that would drive them away from consensus. This approach allows distributed systems to deal with unknown disruptions, without knowledge of the network topology or the numbers and behaviours of the disruptors, a general requirement of other resilient consensus algorithms. A disruptor-agnostic algorithm is particularly suitable to real-world applications where information regarding disruptors or network properties is typically unknown. Faster, tighter, and more robust convergence can be achieved across a range of scenarios with the social dynamics inspired algorithm presented herein, when compared with Mean-Subsequence-Reduced-type methods.

Abstract To measure the effectiveness of learning activities, intensive research works have focused on the process of competency building through the identification of learning stages as well as the setup of related key performance indictors to measure the attainment of specific learning objectives. To organize the learning activities as per the background and skills of each learner, individual learning styles have been identified and measured by several researchers. Despite their importance in personalizing the learning activities, these styles are difficult to implement for large groups of learners. They have also been rarely correlated with each specific learning stage. New approaches are, therefore, needed to intelligently coordinate all the learning activities while self-adapting to the ongoing progress of learning as well as to the specific requirements and backgrounds of learners. To address these issues, we propose in this paper a new framework for an intelligent management of the competency building process during learning. Our framework is based on a recursive spiral Assess-Predict-Oversee-Transit model that is orchestrated by a multi-agent system. This system is particularly responsible of enabling smart transitions between learning stages. It is also responsible of assessing and predicting the process of competency building of the learner and, then, making the right decisions about the learning progress, accordingly. Results of our solution were demonstrated via an Augmented Reality app that we created using the Unity3D engine to train learners on Air Conditioner maintenance.

We consider the controllability of multi-agent dynamical systems modeled by a particular class of bipartite graphs, called chain graphs. Our main focus is related to chain graphs with exactly one repeated degree. We determine all chain graphs with this structural property and derive some properties of their Laplacian eigenvalues and associated eigenvectors. On the basis of the obtained theoretical results, we compute the minimum number of leading agents that make the system in question controllable and locate the leaders in the corresponding graph. Additionaly, we prove that a chain graph with exactly one repeated degree, that is not a star or a regular complete bipartite graph, has the second smallest Laplacian eigenvalue (also known as the algebraic connectivity) in $ (0.8299, 1) $ and we show that the second smallest eigenvalue increases when the number of vertices increases. This result is of a particular interest in control theory, since families of controllable graphs whose algebraic connectivity is bounded from below model the systems with a small risk of power or communication failures.

This paper demonstrates several sufficient frameworks for the multi-cluster flocking behavior of the fractional Cucker–Smale (CS) model. For this, we first employ the Caputo fractional derivative instead of the usual derivative to propose the fractional CS model with the memory effect. Then, using mathematical tools based on fractional calculus, we present suitable sufficient conditions in terms of properly separated initial data close to the multi-cluster, and well-prepared system parameters for the multi-cluster flocking of the fractional system to emerge. Finally, we offer several numerical simulations and compare them with the analytical results.

This paper presents a novel leader-follower consensus control for a particular class of nonlinear multi-agent mechanical systems in the presence of control input constraints and external disturbances which includes robot system dynamics with a wide range of potential applications in industry. In this case, one of the agents is selected as the leader to direct the other agents in such a way that the whole system can reach consensus within certain prescribed performance transient bounds. Due to the presence of disturbances in most practical systems, the effect of limited disturbance in the consensus control method has been investigated, and actuator saturation is included in the design process. A terminal sliding mode control method has been adapted to ensure the stability of the overall system and fast finite-time leader-follower consensus control. The simulation results of the multi-agent nonlinear robot system in MATLAB environment, in different scenarios with simultaneous consideration of actuator saturation and external disturbance, will show the efficiency of the proposed control method.

In recent years, the combination of reinforcement learning and attention mechanisms has attracted an increasing attention in algorithmic research field. Attention mechanisms play an important role in improving the performance of algorithms in reinforcement learning. This paper mainly focuses on the development of attention mechanisms in deep reinforcement learning and examining their applications in the multi-agent reinforcement learning domain. Relevant researches are conducted accordingly. Firstly, the background and development of attention mechanisms and reinforcement learning are introduced, and relevant experimental platforms in this field are also presented. Secondly, classical algorithms of reinforcement learning and attention mechanisms are reviewed and attention mechanism is categorized from different perspectives. Thirdly, practical applications of attention mechanisms in the reinforcement field are sorted out based on three types of tasks including fully cooperative, fully competitive and mixed, with focus on the application in the field of multi-agent. Finally, the improvement of attention mechanisms on reinforcement learning algorithms is summarized. The challenges and future prospects in this field are discussed.

Internet of Things (IoT) technologies are increasingly implemented in buildings as the cost-effective smart sensing infrastructure of building automation systems (BASs). They are also dispersed computing resources for novel distributed optimal control approaches. However, wireless communication networks are critical to fulfill these tasks with the performance influenced by inherent uncertainties in networks, e.g., unpredictable occurrence of link failures. Centralized and hierarchical distributed approaches are vulnerable against link failure, while the robustness of fully distributed approaches depends on the algorithms adopted. This study therefore proposes a fully distributed robust optimal control approach for air-conditioning systems considering uncertainties of communication link in IoT-enabled BASs. The distributed algorithm is adopted that agents know their out-neighbors only. Agents directly coordinate with the connected neighbors for global optimization. Tests are conducted to test and validate the proposed approach by comparing with existing approaches, i.e., the centralized, the hierarchical distributed and the fully distributed approaches. Results show that different approaches are vulnerable against to uncertainties of communication link to different extents. The proposed approach always guarantees the optimal control performance under normal conditions and conditions with link failures, verifying its high robustness. It also has low computation complexity and high optimization efficiency, thus applicable on IoT-enabled BASs.

The operation of multi‐agent systems (MAS) in space necessitates considerations for obstacle avoidance, collision prevention, and connectivity among agents. This coupling conflict, stemming from spatial resource utilization, poses a significant and non‐negligible threat to the safe operation of MAS. This article proposes a consensus control scheme for the time‐varying formation of MAS, incorporating functions for maintaining connectivity, collision avoidance, and obstacle dodging. This scheme effectively constrains the time‐varying formation tracking error within an arbitrarily small range. The considered system model takes a high‐order nonlinear form, with uncertainties and disturbances present in each order. This grants the control scheme with high generality and practicability. By employing barrier Lyapunov function to delineate the safe operational space of MAS, conflicts in spatial resource utilization are avoided. This approach simultaneously fulfills the requirements for connectivity maintenance, collision avoidance, and obstacle dodging in the safe operation of MAS. An additional rotation operator is integrated into the controller to smoothly address the “minima” problem, eliminating the need for external intervention. Gaussian radial basis function is used to estimate the nonlinear terms, uncertainties, and unknown perturbation online in the system. The stability of the MAS under the proposed control scheme is analyzed through Lyapunov function. Finally, numerical simulation results are demonstrated to explain the effectiveness of the control scheme. [ABSTRACT FROM AUTHOR]

This paper explores the group consensus problem of second-order multi-agent systems (MAS) under asynchronous denial-of-service (DoS) attacks. Asynchronous DoS attacks involve the interruption of certain communication links, allowing the MAS to be reimagined as a switching system with a persistent dwell time (PDT). The agents in each group can be divided into three types, which are inter-act agents, intra-act agents with zero in-degree, and other agents. Then, according to the properties of the different agents, suitable agents are pinned. By leveraging the concepts of switching topology and the PDT, a suitable event-triggered control protocol is designed, along with the establishment of conditions to ensure the group consensus of the MAS. Moreover, through the construction of topology-dependent Lyapunov functions, the achievement of group consensus within the MAS under asynchronous DoS attacks is demonstrated. Subsequently, a numerical example is presented to validate the effectiveness of the proposed results. [ABSTRACT FROM AUTHOR]

In Indonesia, the power generation sector is the primary source of carbon emissions, largely due to the heavy reliance on coal-fired power plants, which account for 60% of electricity production. Reducing these emissions is essential to achieve national clean energy transition goals. However, achieving this initiative requires careful consideration, especially regarding the complex interactions among multiple stakeholders in the Indonesian electricity market. The electricity market in Indonesia is characterized by its non-competitive and heavily regulated structure. This market condition often requires the PLN, as the system operator, to address multi-objective and multi-constraint problems, necessitating optimization in the generation dispatch scheduling scheme to ensure a secure, economical, and low-carbon power system operation. This research introduces a multiparadigm approach for GS optimization in a regulated electricity market to support the transition to clean energy. The multiparadigm integrates multi-agent system and system dynamic paradigms to model, simulate, and quantitatively analyze the complex interactions among multiple stakeholders in the Indonesian regulated electricity market. The research was implemented on the Java–Madura–Bali power system using AnyLogic 8 University Researcher Software. The simulation results demonstrate that the carbon policy scheme reduces the system's carbon emissions while increasing the system's cost of electricity. A linear regression for sensitivity analysis was conducted to determine the relationship between carbon policies and the system's cost of electricity. This research offers valuable insights for policymakers to develop an optimal, acceptable, and reasonable power system operation scheme for all stakeholders in the Indonesian electricity market. [ABSTRACT FROM AUTHOR]

A multi-area power system requires coordination to enhance reliability and reduce operating costs. Economic dispatch in such systems is crucial because of the uncertainties associated with variable loads and the increasing penetration of renewable energy sources. This paper presents a hierarchical consensus algorithm designed to determine the economic dispatch in a multi-area power system, accounting for the uncertainties in load and renewable generation. The proposed algorithm, which utilizes distributed agents, operates across three levels. Level 1 coordinates all areas, while levels 2 and 3 form a leader–follower consensus algorithm for overall economic dispatch. Breadth-first search is employed to identify the leader agent within each area. To address the uncertainties in loads and renewable generation, Monte Carlo simulations are performed. The efficacy of the proposed method is validated using the IEEE 39-bus and 118-bus systems, as well as a realistic 1968-bus power system in Taiwan. The traditional equal lambda method is employed to verify that the proposed approach is suitable for multi-area power systems using distributed computation. [ABSTRACT FROM AUTHOR]

Digital Twins (DTs) are emerging as a fundamental brick of engineering Cyber-Physical Systems (CPSs), but their notion is still mostly bound to specific business domains (e.g. manufacturing), goals (e.g. product design), or applications (e.g. the Internet of Things). As such, their value as general purpose engineering abstractions is yet to be fully revealed. In this paper, we relate DTs with agents and multiagent systems, as the latter are arguably the most rich abstractions available for the engineering of complex socio-technical and CPSs, and the former could both fill in some gaps in agent-oriented software engineering and benefit from an agent-oriented interpretation—in a cross-fertilisation journey. [ABSTRACT FROM AUTHOR]

A Digital Twin (DT) is a frequently updated virtual representation of a physical or a digital instance that captures its properties of interest. Incorporating both cyber and physical parts to build a digital twin is challenging due to the high complexity of the requirements that should be addressed and satisfied during the design, implementation and operation. In this context, we introduce the MADTwin (Multi-Agent Digital Twin) framework driven by a Multi-agent Systems (MAS) paradigm and supported by flexible architecture and extendible upper ontology for modelling agent-based digital twins. A comprehensive case study of a smart warehouse supported by multi-robots has been presented to show the feasibility and applicability of this framework. The introduced framework powered by intelligent agents integrated with enabler technologies enabled us to cope with parts of the challenges imposed by modelling and integrating Cyber-Physical Systems (CPS) with digital twins for multi-robots of the smart warehouse. In this framework, different components of CPS (robots) are represented as autonomous physical agents with their digital twin agents in the digital twin environment. Agents act autonomously and cooperatively to achieve their local goals and the objectives of the whole system. Eventually, we discuss the framework's strengths and identify areas of improvement and plans for future work. [ABSTRACT FROM AUTHOR]

This paper presents a novel leader-follower consensus control for a particular class of nonlinear multi-agent mechanical systems in the presence of control input constraints and external disturbances which includes robot system dynamics with a wide range of potential applications in industry. In this case, one of the agents is selected as the leader to direct the other agents in such a way that the whole system can reach consensus within certain prescribed performance transient bounds. Due to the presence of disturbances in most practical systems, the effect of limited disturbance in the consensus control method has been investigated, and actuator saturation is included in the design process. A terminal sliding mode control method has been adapted to ensure the stability of the overall system and fast finite-time leader-follower consensus control. The simulation results of the multi-agent nonlinear robot system in MATLAB environment, in different scenarios with simultaneous consideration of actuator saturation and external disturbance, will show the efficiency of the proposed control method. [ABSTRACT FROM AUTHOR]

This paper is concerned with the secure output consensus problem for the heterogeneous multi-agent systems under the event-triggered scheme in the presence of the denial-of-service attack. Without detecting the attack, the hold-input controller update strategy is adopted when some transmission data may be lost due to the effect of the attack. Based on the tolerable duration of the attack, a novel edge-based event-triggered scheme is developed. The scheme can avoid continuous communication and exclude Zeno behavior. With the aid of the switched system theory, output consensus is preserved. An example shows the effectiveness. • Secure problem for the heterogeneous multi-agent systems is solved in the presence of the denial-of-service attack. • A novel edge-based event-triggered scheme is developed based on the tolerable duration of the attack. • Without detecting the attack, the hold-input controller update strategy is adopted. • Output consensus is preserved with the aid of the switched system theory. [ABSTRACT FROM AUTHOR]

The objective of the invariant ellipsoid method is to minimize the smallest invariant and attractive set of a linear control system operating under the influence of bounded external disturbances. This study extends the application of this method to address the leader-following consensus problem. Initially, a linear control protocol is designed for the multi-agent system in the absence of disturbances. Subsequently, in the presence of bounded disturbances, by employing a similar linear control protocol, a necessary and sufficient condition is introduced to derive the optimal control parameters for the multi-agent system such that the state of followers converges to and remains in a minimal invariant ellipsoid around the state of the leader. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Shanghai Jiaotong University (Science) is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)