Search

This paper investigates a parameter-free H∞ differential game approach for nonlinear active vehicle suspensions. The study accounts for the geometric nonlinearity of the half-car active suspension and the cubic nonlinearity of the damping elements. The nonlinear H∞ control problem is reformulated as a zero-sum game between two players, leading to the establishment of the Hamilton–Jacobi–Isaacs (HJI) equation with a Nash equilibrium solution. To minimize reliance on model parameters during the solution process, an actor–critic framework employing neural networks is utilized to approximate the control policy and value function. An off-policy reinforcement learning method is implemented to iteratively solve the HJI equation. In this approach, the disturbance policy is derived directly from the value function, requiring only a limited amount of driving data to approximate the HJI equation's solution. The primary innovation of this method lies in its capacity to effectively address system nonlinearities without the need for model parameters, making it particularly advantageous for practical engineering applications. Numerical simulations confirm the method's effectiveness and applicable range. The off-policy reinforcement learning approach ensures the safety of the design process. For low-frequency road disturbances, the designed H∞ control policy enhances both ride comfort and stability. [ABSTRACT FROM AUTHOR]

In recent years, with the accelerated development of technology towards automation and intelligence, autonomous decision-making capabilities in unmanned systems are poised to play a crucial role in contemporary aerial two-player zero-sum games (TZSGs). Deep reinforcement learning (DRL) methods enable agents to make autonomous manoeuvring decisions. This paper focuses on current mainstream DRL algorithms based on fundamental tactical manoeuvres, selecting a typical aerial TZSG scenario—within visual range (WVR) combat. We model the key elements influencing the game using a Markov decision process (MDP) and demonstrate the mathematical foundation for implementing DRL. Leveraging high-fidelity simulation software (Warsim v1.0), we design a prototypical close-range aerial combat scenario. Utilizing this environment, we train mainstream DRL algorithms and analyse the training outcomes. The effectiveness of these algorithms in enabling agents to manoeuvre in aerial TZSG autonomously is summarised, providing a foundational basis for further research. [ABSTRACT FROM AUTHOR]

To achieve the intelligent interception of different types of maneuvering evaders, based on deep reinforcement learning, a novel intelligent differential game guidance law is proposed in the continuous action domain. Different from traditional guidance laws, the proposed guidance law can avoid tedious manual settings and save cost efforts. First, the interception problem is transformed into the pursuit–evasion game problem, which is solved by zero-sum differential game theory. Next, the Nash equilibrium strategy is obtained through the Markov game process. To implement the proposed intelligent differential game guidance law, an actor–critic neural network based on deep deterministic policy gradient is constructed to calculate the saddle point of the differential game guidance problem. Then, a reward function is designed, which includes the tradeoffs among guidance accuracy, energy consumption, and interception time. Finally, compared with traditional methods, the interception accuracy of the proposed intelligent differential game guidance law is 99.2%, energy consumption is reduced by 47%, and simulation time is shortened by 1.58 s. All results reveal that the proposed intelligent differential game guidance law has better intelligent decision-making ability. [ABSTRACT FROM AUTHOR]

Unmanned surface pursuit is a complex and challenging engineering problem, especially when conducted by multiple intelligent surface vehicles (ISVs). To enhance the pursuit performance and facilitate strategic interaction during the target pursuit, this paper proposes a novel game theory-based maneuver planning method for pursuit ISVs. Firstly, a specific two-player zero-sum pursuit–evasion game (ZSPEG)-based target-pursuit model is formed. To ensure the vehicles reach a quick consensus, a target-guided relay-pursuit mechanism and the corresponding pursuit payoffs are designed. Meanwhile, under the fictitious play framework, the behavioral pattern and the strategies of the target could be fictitiously learned. Furthermore, mixed-strategy Nash equilibrium (MNE) is employed to determine the motions for the vehicles, the value of which is the best response in the proposed ZSPEG model. Finally, simulations verify the effectiveness of the above methods in multi-ISV surface pursuit. [ABSTRACT FROM AUTHOR]

All the n possible returns on a financial asset are the components of an element of a linear space over R. This paper shows how to transfer all these n possible returns on a one-dimensional straight line. In this research work, two or more than two financial assets are studied. More than two financial assets are always studied in pairs, so they are treated inside the budget set of a given decision-maker. Two univariate financial assets give rise to a bivariate financial asset characterized by a bivariate (two-dimensional) distribution of probability. This research work shows how constrained choices being made by a given decision-maker under conditions of uncertainty and riskiness maximize his utility of an ordinal nature. For this reason, prevision bundles are dealt with. Furthermore, every choice identifies a zero-sum game. Since a specific kind of choice associated with two or more than two objects is investigated, new conceptual and mathematical outcomes related to financial decisions are shown. [ABSTRACT FROM AUTHOR]

This document is an introduction to a special issue on international counter-trafficking. It acknowledges that human trafficking is a complex and fluid concept, and that counter-trafficking efforts often prioritize securitarian, moralistic, and neoliberal concerns. The special issue includes articles that provide valuable insights into the experiences of sex workers, Quran teachers, Nigerian migrant women, and women in counter-trafficking programs in Italy and Spain. These articles challenge dominant narratives about trafficking and highlight the need for more critical approaches to counter-trafficking. The author expresses gratitude to the contributors and the journal for their interest in this important topic. [Extracted from the article]

Network attack and defence games are gradually becoming a new approach through which to study the protection of infrastructure networks such as power grids and transportation networks. Uncertainty factors, such as the subjective decision preferences of attackers and defenders, are not considered in existing attack and defence game studies for infrastructure networks. In this paper, we introduce, respectively, the attacker's and defender's expectation value, rejection value, and hesitation degree of the target, as well as construct an intuitionistic fuzzy goal-based attack and defence game model for infrastructure networks that are based on the maximum connectivity slice size, which is a network performance index. The intuitionistic fuzzy two-player, zero-sum game model is converted into a linear programming problem for solving, and the results are analysed to verify the applicability and feasibility of the model proposed in this paper. Furthermore, different situations, such as single-round games and multi-round repeated games, are also considered. The experimental results show that, when attacking the network, the attacker rarely attacks the nodes with higher importance in the network, but instead pays more attention to the nodes that are not prominent in the network neutrality and median; meanwhile, the defender is more inclined to protect the more important nodes in the network to ensure the normal performance of the network. [ABSTRACT FROM AUTHOR]

With the development of power technology and communication technology, the power grid and power communication network have become interdependent and closely coupled. The load shedding operation of the power grid is an important means to reduce the occurrence of chain faults and ensure the safe and stable operation of the power grid. Based on the transmission of load control services in the communication network, this paper establishes a model for a dual-network coupled cyber physical power system (CPPS). Considering communication edge faults, the associated load capacity of the communication edge and the expected load loss of the power grid are defined. On this basis, the paper proposes a complete information zero-sum game mechanism called "defense attack defense" for communication edge failures, which takes the expected loss of load from the power grid as the benefit. The paper studies the optimal attack and defense game strategies and provides the algorithm implementation process for the three stages of the game. Considering the bandwidth capacity of the communication edge, this paper uses the Dijkstra algorithm or k shortest paths (KSP) algorithm with the cost factor of the communication edge as the weight to plan the main and backup communication channels for multiple load control services. The simulation results show that the game mechanism proposed in this paper can effectively reduce the expected load loss from the power grid and improve the stability of the CPPS. [ABSTRACT FROM AUTHOR]

10.3390/math10081212 8 Ibragimov G., Kuchkarova S., Hasim R.M., Pansera B.A. Differential Game for an Infinite System of Two-Block Differential Equations. References 1 Ibragimov G., Kazimirova R., Pansera B.A. Evasion Differential Game of Multiple Pursuers and One Evader for an Infinite System of Binary Differential Equations. The study of differential games has practical applications in the analysis and resolution of conflict issues by the use of differential equations. [Extracted from the article]

In this paper, the safe optimal control method for continuous-time (CT) nonlinear safety-critical systems with asymmetric input constraints and unmatched disturbances based on the adaptive dynamic programming (ADP) is investigated. Initially, a new non-quadratic form function is implemented to effectively handle the asymmetric input constraints. Subsequently, the safe optimal control problem is transformed into a two-player zero-sum game (ZSG) problem to suppress the influence of unmatched disturbances, and a new Hamilton–Jacobi–Isaacs (HJI) equation is introduced by integrating the control barrier function (CBF) with the cost function to penalize unsafe behavior. Moreover, a damping factor is embedded in the CBF to balance safety and optimality. To obtain a safe optimal controller, only one critic neural network (CNN) is utilized to tackle the complex HJI equation, leading to a decreased computational load in contrast to the utilization of the conventional actor–critic network. Then, the system state and the parameters of the CNN are uniformly ultimately bounded (UUB) through the application of the Lyapunov stability method. Lastly, two examples are presented to confirm the efficacy of the presented approach. [ABSTRACT FROM AUTHOR]

Attacking a naval vessel with multiple missiles is an important way to improve the hit rate of missiles. Missile-borne radars need to complete detection and antijamming tasks to guide missiles, but communication between these radars is often difficult. In this paper, an optimization method based on multi-agent reinforcement learning is proposed for the collaborative detection and antijamming tasks of multiple radars against one naval vessel. We consider the collaborative radars as one player to make their confrontation with the naval vessel a two-person zero-sum game. With temporal constraints of the radar's and jammer's recognition and preparation interval, the game focuses on taking a favorable position at the end of the confrontation. It is assumed the total jamming capability of a shipborne jammer is constant and limited, and the shipborne jammer allocates the jamming capability in the radar's direction according to the radar threat assessment result and its probability of successful detection. The radars work collaboratively through prior centralized training and obtain a good performance by decentralized execution. The proposed method can make radars collaborate to detect the naval vessel, rather than only considering the detection result of each radar itself. Experimental results show that the proposed method in this paper is effective, improving the winning probability to 10% and 25% in the two-radar and four-radar scenarios, respectively. [ABSTRACT FROM AUTHOR]

This work proposes a method for the intelligent deployment of distributed flexible AC transmission system (D-FACTS) devices. In recent years, in the field of moving target defense (MTD) strategies to detect coordinated cyber–physical attacks (CCPAs), establishing how to deploy D-FACTS devices has become an important research point. Although some research results have been proposed, the obtained deployment solutions are unintelligent due to not carefully considering smart attackers' behaviors. A method for achieving the intelligent deployment of D-FACTS devices is proposed in this paper. First, the basic concept of corrupting CCPAs is summarized; second, based on considering practical constraints and the basic concept, a protected transmission line set is confirmed; and third, a zero-sum game model is formulated, and a robust Nash equilibrium solution is computed. Due to the game's characteristics, this solution reflects the smart attackers' sense of action. Relying on the solution, those lines that are most likely to be tripped form a new protected transmission line set. Finally, a comprehensive algorithm using a metric proposed in previous studies is proposed for finding an intelligent solution for the deployment of D-FACTS devices. We validated our results through extensive simulations using IEEE 14-bus, 30-bus, and 118-bus power systems provided by MATPOWER and the real-world load profiles from New York State. Our work, in tracking the targets that attackers are most likely to attack, opens up new ideas for the intelligent deployment of D-FACTS devices. [ABSTRACT FROM AUTHOR]

In the field of computer intelligence, it has always been a challenge to construct an agent model that can be adapted to various complex tasks. In recent years, based on the planning algorithm of Monte Carlo tree search (MCTS), a new idea has been proposed to solve the AI problems of two-player zero-sum games such as chess and Go. However, most of the games in the real environment rely on imperfect information, so it is impossible to directly use the normal tree search planning algorithm to construct a decision-making model. Mahjong, which is a popular multiplayer game with a long history in China, attracts great attention from AI researchers because it contains a large game state space and a lot of hidden information. In this paper, we utilize an agent learning approach that leverages deep learning, reinforcement learning, and dropout learning techniques to implement a Mahjong AI game agent. First, we improve the state transition of the tree search based on the learned MDP model, the player position variable and transition information are introduced into the tree search algorithm to construct a multiplayer search tree. Then, the model training based on a deep reinforcement learning method ensures the stable and sustainable training process of the learned MDP model. Finally, we utilize the strategy data generated by the tree search and use the dropout learning method to train the normal decision-making agent. The experimental results demonstrate the efficiency and stability performance of the agent trained by our proposed method compared with existing agents in terms of test data accuracy, tournament ranking performance, and online match performance. The agent plays against human players and acts like real humans. [ABSTRACT FROM AUTHOR]

The research question in this article concerns how a competitive environment affects the learner's (officer cadet's) personal leadership development and their relationship to their team and with future civilian foundations. More specifically, what are the possible learning effects of the 'hidden' curriculum? This article investigates how more than 250 years of leadership education provides new army officers with new skills and how this environment may affect the cadets' leadership training. The article builds on ethnographic data gathered during the three-year education programme in most of the relevant practical locations and contexts. Data were collected using both interviews and a questionnaire. Regarding trust in their learning environment, cadets reported a mean score of 2.83 on a 1 (low trust) to 5 (high trust) Likert scale, underpinning interview data regarding the lack of trust in the Norwegian Military Academy (NMA) and in their fellow cadets. Cadets also pointed out that competition hindered their learning (mean = 2.50). These findings are interpreted in relation to possible negative effects stemming from internal competition and the evaluation system as a whole. The overall output of this system is a zero-sum game, and thus affects evaluative practices and learning processes. This study is of relevance to higher education officers responsible for designing learning environments. [ABSTRACT FROM AUTHOR]

This paper is concerned with the control law synthesis for robot manipulators, which guarantees that the effect of the sensor faults is kept under a permissible level, and ensures the stability of the closed-loop system. Based on Lyapunov's stability analysis, the conditions that enable the application of the simple bisection method in the optimization procedure were derived. The control law, with certain properties that make the construction of the Lyapunov function much easier—and, thus, the determination of stability conditions—was considered. Furthermore, the optimization problem was formulated as a class of problem in which minimization and maximization of the same performance criterion were simultaneously carried out. The algorithm proposed to solve the related zero-sum differential game was based on Newton's method with recursive matrix relations, in which the first- and second-order derivatives of the objective function are calculated using hyper-dual numbers. The results of this paper were evaluated in simulation on a robot manipulator with three degrees of freedom. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) have been used for many applications with overwhelming success. The training process of these models is complex, involving a zero-sum game between two neural networks trained in an adversarial manner. Thus, to use GANs, researchers and developers need to answer the question: "Is the GAN sufficiently trained?". However, understanding when a GAN is well trained for a given problem is a challenging and laborious task that usually requires monitoring the training process and human intervention for assessing the quality of the GAN generated outcomes. Currently, there is no automatic mechanism for determining the required number of epochs that correspond to a well-trained GAN, allowing the training process to be safely stopped. In this paper, we propose AutoGAN, an algorithm that allows one to answer this question in a fully automatic manner with minimal human intervention, being applicable to different data modalities including imagery and tabular data. Through an extensive set of experiments, we show the clear advantage of our solution when compared against alternative methods, for a task where the GAN outputs are used as an oversampling method. Moreover, we show that AutoGAN not only determines a good stopping point for training the GAN, but it also allows one to run fewer training epochs to achieve a similar or better performance with the GAN outputs. [ABSTRACT FROM AUTHOR]

In this paper, an event-triggered adaptive dynamic programming (ADP) method is proposed to deal with the H ∞ problem with unknown dynamic and constrained input. Firstly, the H ∞ -constrained problem is regarded as the two-player zero-sum game with the nonquadratic value function. Secondly, we develop the event-triggered Hamilton–Jacobi–Isaacs(HJI) equation, and an event-triggered ADP method is proposed to solve the HJI equation, which is equivalent to solving the Nash saddle point of the zero-sum game. An event-based single-critic neural network (NN) is applied to obtain the optimal value function, which reduces the communication resource and computational cost of algorithm implementation. For the event-triggered control, a triggering condition with the level of disturbance attenuation is developed to limit the number of sampling states, and the condition avoids Zeno behavior by proving the existence of events with minimum triggering interval. It is proved theoretically that the closed-loop system is asymptotically stable, and the critic NN weight error is uniformly ultimately boundedness (UUB). The learning performance of the proposed algorithm is verified by two examples. [ABSTRACT FROM AUTHOR]

This paper presents an approach for the solution of a zero-sum differential game associated with a nonlinear state-feedback H ∞ control problem. Instead of using the approximation methods for solving the corresponding Hamilton–Jacobi–Isaacs (HJI) partial differential equation, we propose an algorithm that calculates the explicit inputs to the dynamic system by directly performing minimization with simultaneous maximization of the same objective function. In order to achieve numerical robustness and stability, the proposed algorithm uses: quasi-Newton method, conjugate gradient method, line search method with Wolfe conditions, Adams approximation method for time discretization and complex-step calculation of derivatives. The algorithm is evaluated in computer simulations on examples of first- and second-order nonlinear systems with analytical solutions of H ∞ control problem. [ABSTRACT FROM AUTHOR]

This study aims to solve the issue of the safe navigation of autonomous underwater vehicles (AUVs) in an unknown underwater environment. AUV will encounter canyons, rocks, reefs, fish, and underwater vehicles that threaten its safety during underwater navigation. A game-based soft actor–critic (GSAC) path planning method is proposed in this study to improve the adaptive capability of autonomous planning and the reliability of obstacle avoidance in the unknown underwater environment. Considering the influence of the simulation environment, the obstacles in the simulation environment are regarded as agents and play a zero-sum game with the AUV. The zero-sum game problem is solved by improving the strategy of AUV and obstacles, so that the simulation environment evolves intelligently with the AUV path planning strategy. The proposed method increases the complexity and diversity of the simulation environment, enables AUV to train in a variable environment specific to its strategy, and improves the adaptability and convergence speed of AUV in unknown underwater environments. Finally, the Python language is applied to write an unknown underwater simulation environment for the AUV simulation testing. GSAC can guide the AUV to the target point in the unknown underwater environment while avoiding large and small static obstacles, canyons, and small dynamic obstacles. Compared with the soft actor–critic(SAC) and the deep Q-network (DQN) algorithm, GSAC has better adaptability and convergence speed in the unknown underwater environment. The experiments verifies that GSAC has faster convergence, better stability, and robustness in unknown underwater environments. [ABSTRACT FROM AUTHOR]

Location Privacy Preservation (LPP) in Wireless Sensor Networks (WSNs) during the era of the Internet of things and smart systems is a critical element in the success of WSNs. LPP in WSN can be stated as: given a WSN with an adversary aiming to unravel the location of critical nodes of a WSN, the goal of the WSN manager is to enshroud the location of the critical nodes via routing and/or encryption mechanisms. Typical research in the LPP of WSN routing involves developing and/or estimating the performance of a fixed routing protocol under a given attack mechanism. Motivated by advancements in network softwarization, in this work, we propose an approach where the WSN manager as well as the WSN adversary can deploy multiple routing and attack mechanisms, respectively. Initially, the proposed approach is formulated as a repeated two-player zero-sum game. The formulation is further extended to handle multiple objectives and incomplete information in the game matrix. In this work, the multiple objectives are handled via the epsilon constraint method. The presence of incomplete information in the formulation is modeled as interval based uncertainty. To sum, the proposed formulation ultimately boils down to linear programming problems, which can be efficiently solved. Numerical case studies to showcase the applicability of the proposed approach are illustrated in this work. Finally, discussion on obtaining the required data from any given WSN, discussion and interpretation of the formulation's results, and future research direction of the current work is presented. [ABSTRACT FROM AUTHOR]

This paper proposes a reinforcement learning (RL) algorithm for the security problem of state estimation of cyber-physical system (CPS) under denial-of-service (DoS) attacks. The security of CPS will inevitably decline when faced with malicious cyber attacks. In order to analyze the impact of cyber attacks on CPS performance, a Kalman filter, as an adaptive state estimation technology, is combined with an RL method to evaluate the issue of system security, where estimation performance is adopted as an evaluation criterion. Then, the transition of estimation error covariance under a DoS attack is described as a Markov decision process, and the RL algorithm could be applied to resolve the optimal countermeasures. Meanwhile, the interactive combat between defender and attacker could be regarded as a two-player zero-sum game, where the Nash equilibrium policy exists but needs to be solved. Considering the energy constraints, the action selection of both sides will be restricted by setting certain cost functions. The proposed RL approach is designed from three different perspectives, including the defender, the attacker and the interactive game of two opposite sides. In addition, the framework of Q-learning and state–action–reward–state–action (SARSA) methods are investigated separately in this paper to analyze the influence of different RL algorithms. The results show that both algorithms obtain the corresponding optimal policy and the Nash equilibrium policy of the zero-sum interactive game. Through comparative analysis of two algorithms, it is verified that the differences between Q-Learning and SARSA could be applied effectively into the secure state estimation in CPS. [ABSTRACT FROM AUTHOR]

Featured Application: This proposed method is suitable for the cooperative consensus control of various homogeneous Multiple Autonomous Unmanned linear systems, such as an underwater robot swarm and aerial UAV swarm. This paper addresses the distributed optimal decoupling synchronous control of multiple autonomous unmanned linear systems (MAUS) subject to complex network dynamic coupling. The leader–follower mechanism based on neighborhood error dynamics is established and the network coupling term is regarded as the external disturbance to realize the decoupling cooperative control of each agent. The Bounded L2-Gain problem for the network coupling term is formulated into a multi-player zero-sum differential game. It is shown that the solution to the multi-player zero-sum differential game requires the solution to coupled Hamilton–Jacobi (HJ) equations. The coupled HJ equations are transformed into an algebraic Riccati equation (ARE), which can be solved to obtain the Nash equilibrium of a multi-player zero-sum game. It is shown that the bounded L2-Gain for coupling attenuation can be realized by applying the zero-sum game solution as the control protocol and the ultimately uniform boundedness (UUB) of a local neighborhood error vector under conservative conditions is proved. A simulation example is provided to show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we consider the two-player state and control path-dependent stochastic zero-sum differential game. In our problem setup, the state process, which is controlled by the players, is dependent on (current and past) paths of state and control processes of the players. Furthermore, the running cost of the objective functional depends on both state and control paths of the players. We use the notion of non-anticipative strategies to define lower and upper value functionals of the game, where unlike the existing literature, these value functions are dependent on the initial states and control paths of the players. In the first main result of this paper, we prove that the (lower and upper) value functionals satisfy the dynamic programming principle (DPP), for which unlike the existing literature, the Skorohod metric is necessary to maintain the separability of càdlàg (state and control) spaces. We introduce the lower and upper Hamilton–Jacobi–Isaacs (HJI) equations from the DPP, which correspond to the state and control path-dependent nonlinear second-order partial differential equations. In the second main result of this paper, we show that by using the functional Itô calculus, the lower and upper value functionals are viscosity solutions of (lower and upper) state and control path-dependent HJI equations, where the notion of viscosity solutions is defined on a compact κ -Hölder space to use several important estimates and to guarantee the existence of minimum and maximum points between the (lower and upper) value functionals and the test functions. Based on these two main results, we also show that the Isaacs condition and the uniqueness of viscosity solutions imply the existence of the game value. Finally, we prove the uniqueness of classical solutions for the (state path-dependent) HJI equations in the state path-dependent case, where its proof requires establishing an equivalent classical solution structure as well as an appropriate contradiction argument. [ABSTRACT FROM AUTHOR]

The paper presents an evolutionary model, based on the assumption that agents may revise their current strategies if they previously failed to attain the maximum level of potential payoffs. We offer three versions of this reflexive mechanism, each one of which describes a distinct type: spontaneous agents, rigid players, and 'satisficers'. We use simulations to examine the performance of these types. Agents who change their strategies relatively easily tend to perform better in coordination games, but antagonistic games generally lead to more favorable outcomes if the individuals only change their strategies when disappointment from previous rounds surpasses some predefined threshold. [ABSTRACT FROM AUTHOR]