Search

A generalized class of the discrete game of timing is solved as a finite zero-sum game defined on a symmetric lattice of the unit square. The game is a progressive discrete silent duel whose kernel is skew-symmetric, and the players, referred to as duelists, have identical shooting accuracy functions featured with an accuracy proportionality factor a and a power constant β describing nonlinearity of the shooting accuracy. As the duel starts, time moments of possible shooting become denser by a geometric progression. Apart from the duel beginning and end time moments, every following time moment is the partial sum of the respective geometric series. Due to the skew-symmetry, both the duelists have the same optimal strategies and the game optimal value is 0. As the duelist has a single bullet, there is no reason for considering solutions in mixed strategies, if any, with non-singleton supports. If a ⩾ 2 β − 1 , the duelist's optimal strategy is the middle of the duel time span; otherwise, the duel solution may be not a pure strategy solution. The case of quadratic accuracy for β = 2 is further considered, which is reasoned by that linearly developing real-time processes are quite rare, and the quadratic accuracy is a non-linearity pattern that makes an interaction model more reliable by slightly diminishing the duelist's confidence. Thus, the case of a < 2 β − 1 is thoroughly studied for pure strategy solutions. A boundary case of a is found, by which the duel has four pure strategy solutions which are of the time moment preceding the duel end moment and the duel end moment itself. • Discrete game of timing is solved as a finite zero-sum game on a symmetric lattice. • Duelist is featured with shooting accuracy proportionality and nonlinearity factors. • The breaking point of the accuracy is found above which pure strategy solutions exist. • Pure strategy solutions are found below the breaking point for the quadratic accuracy. [ABSTRACT FROM AUTHOR]

The potential emergence of artificial general intelligence (AGI) systems has sparked intense debate among researchers, policymakers, and the public due to their potential to surpass human intelligence in all domains. This note argues that for an AI to be considered "general", it should achieve superhuman performance not only in zero-sum games but also in general-sum games, where winning or losing is not clearly defined. In this note, I propose a game-theoretic framework that captures the strategic interactions between a representative human agent and a potential superhuman machine agent. Four assumptions underpin this framework: Superhuman Machine, Machine Strategy, Rationality, and Strategic Unpredictability. The main result is an impossibility theorem, establishing that these assumptions are inconsistent when taken together, but relaxing any one of them results in a consistent set of assumptions. This note contributes to a better understanding of the theoretical context that can shape the development of superhuman AI. • A game between a human vs an Artificial General Intelligence system is introduced. • An AI system is general if it is successful in both zero-sum and general-sum games. • Assumptions are Superhuman AI, Machine Strategy, Rationality, and Unpredictability. • An impossibility theorem shows that these assumptions lead to inconsistency. • The result is tight; relaxing any of the assumptions resolves the inconsistency. [ABSTRACT FROM AUTHOR]

The well-known H ∞ control design gives robustness to a controller by rejecting perturbations from the external environment, which is difficult to do for completely unknown affine nonlinear systems. Accordingly, the immediate objective of this paper is to develop an on-line real-time synergetic learning algorithm, so that a data-driven H ∞ controller can be received. By converting the H ∞ control problem into a two-player zero-sum game, a model-free Hamilton–Jacobi–Isaacs equation (MF-HJIE) is first derived using off-policy reinforcement learning, followed by a proof of equivalence between the MF-HJIE and the conventional HJIE. Next, by applying the temporal difference to the MF-HJIE, a synergetic evolutionary rule with experience replay is designed to learn the optimal value function, the optimal control, and the worst perturbation, that can be performed on-line and in real-time along the system state trajectory. It is proven that the synergistic learning system constructed by the system plant and the evolutionary rule is uniformly ultimately bounded. Finally, simulation results on an F16 aircraft system and a nonlinear system back up the tractability of the proposed method. • A model-free Hamiltonian–Jacobi–Isaacs equation is derived for H ∞ control problems. • A real-time on-line evolutionary rule for tuning neural network weights is developed. • The system and neural network approximation errors achieve synergetic learning. [ABSTRACT FROM AUTHOR]

In this paper, a novel parallel learning framework is developed to solve zero-sum games for discrete-time nonlinear systems. Briefly, the purpose of this study is to determine a tentative function according to the prior knowledge of the value iteration (VI) algorithm. The learning process of the parallel controllers can be guided by the tentative function. That is to say, the neighborhood of the optimal cost function can be compressed within a small range via two typical exploration policies. Based on the parallel learning framework, a novel dichotomy VI algorithm is established to accelerate the learning speed. It is shown that the parallel controllers will converge to the optimal policy from contrary initial policies. Finally, two typical systems are used to demonstrate the learning performance of the constructed dichotomy VI algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we present two-timescale neurodynamic optimization approaches to distributed minimax optimization. We propose four multilayer recurrent neural networks for solving four different types of generally nonlinear convex–concave minimax problems subject to linear equality and nonlinear inequality constraints. We derive sufficient conditions to guarantee the stability and optimality of the neural networks. We demonstrate the viability and efficiency of the proposed neural networks in two specific paradigms for Nash-equilibrium seeking in a zero-sum game and distributed constrained nonlinear optimization. [ABSTRACT FROM AUTHOR]

In this paper, a novel adaptive critic control method is designed to solve an optimal H ∞ tracking control problem for continuous nonlinear systems with nonzero equilibrium based on adaptive dynamic programming (ADP). To guarantee the finiteness of a cost function, traditional methods generally assume that the controlled system has a zero equilibrium point, which is not true in practical systems. In order to overcome such obstacle and realize H ∞ optimal tracking control, this paper proposes a novel cost function design with respect to disturbance, tracking error and the derivative of tracking error. Based on the designed cost function, the H ∞ control problem is formulated as two-player zero-sum differential games, and then a policy iteration (PI) algorithm is proposed to solve the corresponding Hamilton–Jacobi–Isaacs (HJI) equation. In order to obtain the online solution to the HJI equation, a single-critic neural network structure based on PI algorithm is established to learn the optimal control policy and the worst-case disturbance law. It is worth mentioning that the proposed adaptive critic control method can simplify the controller design process when the equilibrium of the systems is not zero. Finally, simulations are conducted to evaluate the tracking performance of the proposed control methods. [ABSTRACT FROM AUTHOR]

• The study explores changes in charitable giving after a major domestic disaster - Hurricane Sandy. • Total donations increase in the immediate aftermath of Hurricane Sandy. • Donations for Sandy relief are, on average, not substituted with similar projects. • Donations for other disaster recovery projects increase in the immediate aftermath of Hurricane Sandy. • In affected regions, these changes in charitable giving appear to be more pronounced than in regions further away. This paper re-opens the discussion of whether new demand for charitable aid generates additional giving or is ultimately a zero-sum game. I employ the exogenous variation in donations provided by Hurricane Sandy and a unique data set from a large crowdfunding platform to analyze giving patterns around the disaster. The study consists of two parts. The first part examines whether a major disaster like Hurricane Sandy increases the altruism budget and if and how donors substitute their donations to the victims of the disaster. The second part focuses on the disaster experience and whether the altruism budget and substitution differ with the proximity to the disaster and resulting damage. The findings indicate that the altruism budget increases in line with earlier studies and that not only victims of Hurricane Sandy, but also victims of other major disasters benefit from this increase at least in the immediate aftermath. The analyses provide no evidence for substitution with similar projects. Giving for victims of Hurricane Sandy and other disasters seems to be slightly more pronounced in affected regions relative to the control group of regions further away. [ABSTRACT FROM AUTHOR]

In this paper, we present a Q-learning framework for solving finite-horizon zero-sum game problems involving the H ∞ control of linear system without knowing the dynamics. Research in the past mainly focused on solving problems in infinite horizon with completely measurable state. However, in the practical engineering, the system state is not always directly accessible, and it is difficult to solve the time-varying Riccati equation associated with the finite-horizon setting directly either. The main contribution of the proposed model-free algorithm is to determine the optimal output feedback policies without measurement state in finite-horizon setting. To achieve this goal, we first describe the Q-function caused by finite-horizon problems in the context of state feedback, then we parameterize the Q-functions as input–output vectors functions. Finally, the numerical examples on aircraft dynamics demonstrate the algorithm's efficiency. [ABSTRACT FROM AUTHOR]

We study zero-sum stochastic games for controlled discrete time Markov chains with risk-sensitive average cost criterion with countable/compact state space and Borel action spaces. The payoff function is nonnegative and possibly unbounded for countable state space case and for compact state space case it is a real-valued and bounded function. For countable state space case, under a certain Lyapunov type stability assumption on the dynamics we establish the existence of the value and a saddle point equilibrium. For compact state space case we establish these results without any Lyapunov type stability assumptions. Using the stochastic representation of the principal eigenfunction of the associated optimality equation, we completely characterize all possible saddle point strategies in the class of stationary Markov strategies. Also, we present and analyze an illustrative example. [ABSTRACT FROM AUTHOR]

In this paper, a synergetic learning structure-based neuro-optimal fault tolerant control (SLSNOFTC) method is proposed for unknown nonlinear continuous-time systems with actuator failures. Under the framework of the synergetic learning structure (SLS), the optimal control input and the actuator failure are viewed as two subsystems. Then, the fault tolerant control (FTC) problem can be regarded as a two-player zero-sum differential game according to the game theory. A radial basis function neural network-based identifier, which uses the measured input/output data, is constructed to identify the completely unknown system dynamics. To develop the SLSNOFTC method, the Hamilton–Jacobi–Isaacs equation is solved by an asymptotically stable critic neural network (ASCNN) which is composed of cooperative adaptive tuning laws. Besides, with the help of the Lyapunov stability analysis, the identification error, the weight error of ASCNN, and all signals of closed-loop system are guaranteed to be converged to zero asymptotically, rather than uniformly ultimately bounded. Numerical simulation examples further verify the effectiveness and reliability of the proposed method. [ABSTRACT FROM AUTHOR]

This paper aims at solving a stochastic two-player zero-sum Nash game problem studied in Singh and Lisser (2019). The main contribution of our paper is that we model this game problem as a dynamical neural network (DNN for short). In this paper, we show that the saddle point of this game problem is the equilibrium point of the DNN model, and we study the globally asymptotically stable of the DNN model. In our numerical experiments, we present the time-continuous feature of the DNN model and compare it with the state-of-the-art convex solvers, i.e., Splitting conic solver (SCS for short) and Cvxopt. Our numerical results show that our DNN method has two advantages in dealing with this game problem. Firstly, the DNN model can converge to a better optimal point. Secondly, the DNN method can solve all problems, even when the problem size is large. [ABSTRACT FROM AUTHOR]

For industrial processes with external disturbance and actuator failure, off-policy reinforcement learning-based novel model-free minmax fault-tolerant control is proposed in this paper to solve H ∞ fault-tolerant tracking control problem. An augmented model equivalent to the original system is constructed, and the state of the new augmented model is composed of state increment and tracking error of the original system. The original H ∞ fault-tolerant tracking problem was transformed into the linear quadratic zero-sum game problem by establishing performance index function, and the Game Algebraic Riccati Equation (GARE) was established. Then Q function was introduced and the Off-policy reinforcement learning algorithm was designed. Different from the traditional model-based fault-tolerant control method, the proposed algorithm does not need the knowledge of system dynamics, and it can learn from the measured data of the system trajectory to solve the GARE. In addition, it is proved that the probing noise added to satisfy the persistent excitation condition does not cause bias. A simulation example of injection molding process is used to verify the effectiveness of the proposed algorithm. • A new off-policy reinforcement learning algorithm based novel model-free minmax fault-tolerant control is proposed. • The optimal control strategy is transformed into minmax zero-sum game method. • Off-policy Q-learning algorithm is used to solve the optimal control strategy and the worst external disturbance. [ABSTRACT FROM AUTHOR]

The infinite-horizon zero-sum game of a linear system can be resorted to solve a Game algebraic Riccati equation (GARE) with indefinite quadratic term. Double-loop policy iteration algorithm is often used to find the solution of such GARE, but its calculation is usually time-consuming. In this work, we propose a novel model-based single-loop policy iteration algorithm to solve GARE and the convergence of the algorithm is guaranteed by the boundness of the iterative sequence and the comparison result. Furthermore, we devise a data-driven single-loop policy iteration algorithm for solving linear zero-sum games, without requiring the knowledge of system dynamics. Compared to the existing Newton's single-loop methods, the initialization of our algorithms is significantly relaxed and easier to implement. Two numerical examples are included to illustrate the proposed algorithms. [ABSTRACT FROM AUTHOR]

• A data-driven waterlogging risk mapping method was proposed based on long-tail learning and game theory. • A novel F (α , β) score was defined to both consider precision and recall and trade-off equity and efficiency in public emergency resource allocation. • A framework for evaluating and enhancing urban waterlogging risk mitigation capabilities was proposed, targeting nonprofit operations and government bailouts. Because disaster reduction resources are always constrained, risk mitigation measures and policies need to be weighted toward efficiency or equity. Greater efficiency focuses on high-risk areas, and greater equity focuses on identifying additional potential risk areas. To address this efficiency-equity tradeoff decision, a data-driven urban waterlogging risk mitigation capability evaluation and improving framework was proposed for nonprofit operations and government bailouts. First, a novel evaluation index was defined, the F (α , β) score, to consider both precision and recall in the binary classification tasks and to seek an effective efficiency-equity trade-off, after which a two-person, zero-sum matrix game model was developed to calculate the α and β parameters and select the highest valued optimal model. Then, using this model, a risk probability map was generated for Wuhan, China, and data envelopment analysis was used to conduct a risk mitigation capability evaluation and identify the needed policy improvements. It was found that: (1) vulnerability was the primary risk-related factor, followed by adaptability and restorability; (2) more importance should be attached to equity than efficiency when implementing risk reduction capability evaluations, that is, authorities should reduce resource investments in extremely high-risk DEA-inefficient areas and identify a greater number of potential risk areas; (3) there were around 1.55% of extremely high-risk areas, which were mainly concentrated in the Jianghan and Wuchang districts and around 9% of high-risk areas, mainly located in the Hongshan district; and (4) northeastern and southwestern Jianghan district could be a valuable reference for urban waterlogging management and disaster reduction. However, excessive investment in transportation infrastructure and construction in DEA inefficient areas should be appropriately reduced. [ABSTRACT FROM AUTHOR]

In this paper, the zero-sum game problem for linear discrete-time Markov jump systems is solved by two novel model-free reinforcement Q -learning algorithms, on-policy Q -learning and off-policy Q -learning. Firstly, under the framework of the zero-sum game, the game-coupled algebraic Riccati equation is derived. On this basis, subsystem transformation technology is employed to decouple the jumping modes. Then, a model-free on-policy Q -learning algorithm is introduced in the zero-sum game architecture to obtain the optimal control gain by measured system data. However, the probing noise will produce biases in on-policy algorithm. Thus, an off-policy Q -learning algorithm is proposed to eliminate the effect of probing noise. Subsequently, convergence is discussed for the proposed methods. Finally, an inverted pendulum system is employed to verify the validity of the proposed methods. • The zero-sum game problem for linear discrete-time Markov jump systems is addressed with the reinforcement learning method. • A novel model-free parallel off-policy Q-learning method is developed, which is not affected by probing noise. • The subsystem transformation technology is employed decoupling the jumping modes to solve the game-coupled algebraic Riccati equation. • The convergence of the proposed methods, on-policy and off-policy, for Markov jump systems are both discussed in this paper. [ABSTRACT FROM AUTHOR]

This paper investigates the optimal tracking issue for continuous-time (CT) nonlinear asymmetric constrained zero-sum games (ZSGs) by exploiting the neural critic technique. Initially, an improved algorithm is constructed to tackle the tracking control problem of nonlinear CT multiplayer ZSGs. Also, we give a novel nonquadratic function to settle the asymmetric constraints. One thing worth noting is that the method used in this paper to solve asymmetric constraints eliminates the strict restriction on the control matrix compared to the previous ones. Further, the optimal controls, the worst disturbances, and the tracking Hamilton–Jacobi–Isaacs equation are derived. Next, a single critic neural network is built to estimate the optimal cost function, thus obtaining the approximations of the optimal controls and the worst disturbances. The critic network weight is updated by the normalized steepest descent algorithm. Additionally, based on the Lyapunov method, the stability of the tracking error and the weight estimation error of the critic network is analyzed. In the end, two examples are offered to validate the theoretical results. [ABSTRACT FROM AUTHOR]

We investigate the interplay between forward uncertain difference equations and backward stochastic difference equations, which belong to distinct mathematical frameworks: uncertainty theory and probability theory, respectively. By establishing the connections between uncertain expectation, probabilistic expectation, and chance expectation, we develop the chance theory framework for analyzing uncertain stochastic hybrid zero-sum games constrained by both forward uncertain difference equations and backward stochastic difference equations. Through an equivalent transformation, we convert these games into deterministic difference equation solving problems. By backward-solving the difference equations and applying mathematical induction, analytical expressions for the equilibrium solutions of two types of uncertain stochastic hybrid zero-sum games are obtained. Furthermore, we explore solution methods for zero-sum games with different operational orders of uncertain and probabilistic expectations and highlight the fundamental differences through numerical examples. • Present uncertain stochastic hybrid zero-sum games of forward and backward systems. • Chance theory framework aids analysis of uncertain stochastic hybrid zero-sum games. • Transforms these games into deterministic difference equation solving problems. • Explores solution methods for games with different expectation operational orders. • Provides numerical examples to illustrate effectiveness of solution methods. [ABSTRACT FROM AUTHOR]

In this paper, an event-triggered tracking optimization control method is proposed for nonlinear uncertain systems with H ∞ discounted cost. First, the linearization method using the infinite series expansion theorem is given. And the tracking control design problem is solved by appropriate state augment. Secondly, the zero sum differential game strategy is introduced to address the control design problem of uncertain systems with event-triggered inputs. The actor-critic-disturbance networks framework is used to approximate the optimization of event- triggered controller strategy, optimization costs, and the disturbance strategy, respectively. Due to the consideration of discrete time and continuous time dynamics, we put the impulsive system method to use and choose appropriate Lyapunov functions to prove stability and boundedness, at the same time, there will be no infinite triggering situation. Finally, the effectiveness of the proposed scheme is demonstrated by an example of a practical tractor system with n trailers through simulation compared with another typical design method. [ABSTRACT FROM AUTHOR]

In this paper, an adjustable Q-learning scheme is developed to solve the discrete-time nonlinear zero-sum game problem, which can accelerate the convergence rate of the iterative Q-function sequence. First, the monotonicity and convergence of the iterative Q-function sequence are analyzed under some conditions. Moreover, by employing neural networks, the model-free tracking control problem can be overcome for zero-sum games. Second, two practical algorithms are designed to guarantee the convergence with accelerated learning. In one algorithm, an adjustable acceleration phase is added to the iteration process of Q-learning, which can be adaptively terminated with convergence guarantee. In another algorithm, a novel acceleration function is developed, which can adjust the relaxation factor to ensure the convergence. Finally, through a simulation example with the practical physical background, the fantastic performance of the developed algorithm is demonstrated with neural networks. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a non-linear Snell envelope which at each time represents the maximal value that can be achieved by stopping a BSDE with constrained jumps. We establish the existence of the Snell envelope by employing a penalization technique and the primary challenge we encounter is demonstrating the regularity of the limit for the scheme. Additionally, we relate the Snell envelope to a finite horizon, zero-sum stochastic differential game, where one player controls a path-dependent stochastic system by invoking impulses, while the opponent is given the opportunity to stop the game prematurely. Importantly, by developing new techniques within the realm of control randomization, we demonstrate that the value of the game exists and is precisely characterized by our non-linear Snell envelope. [ABSTRACT FROM AUTHOR]

In an adversarial environment, a hostile player performing a task may behave like a non-hostile one in order not to reveal its identity to an opponent. To model such a scenario, we define identity concealment games: zero-sum stochastic reachability games with a zero-sum objective of identity concealment. To measure the identity concealment of the player, we introduce the notion of an average player. The average player's policy represents the expected behavior of a non-hostile player. We show that there exists an equilibrium policy pair for every identity concealment game and give the optimality equations to synthesize an equilibrium policy pair. If the player's opponent follows a non-equilibrium policy, the player can hide its identity better. For this reason, we study how the hostile player may learn the opponent's policy. Since learning via exploration policies would quickly reveal the hostile player's identity to the opponent, we consider the problem of learning a near-optimal policy for the hostile player using the game runs collected under the average player's policy. Consequently, we propose an algorithm that provably learns a near-optimal policy and give an upper bound on the number of sample runs to be collected. [ABSTRACT FROM AUTHOR]

Sea lice are copepod ectoparasites of major importance in salmonid aquaculture. Under controlled challenge testing, individual lice count has moderate heritability and has been suggested as a trait in genetic selection for parasite control, under the assumption that selection for reduced individual parasite burden provides group-level protection against the parasite. Recent studies indicate that genetic variation of lice count in Atlantic salmon is mostly explained by variation in initial infestation, rather than ability to limit parasite burden after infestation. Results from a selection experiment are presented, with two Atlantic salmon (Salmo salar) lines divergently selected for low/high parasite burden, showing substantial between-strain difference under common-garden testing. An experiment was performed where the final generation of the divergently selected salmon lines were challenge tested separately to assess potential for group-level protection against sea lice. The results showed that, despite the two groups being clearly different under common garden testing, the line difference was not significant when tested separately, i.e., no evidence for group-level protection. However, in a follow-up experiment, using a more realistic lice challenge model under water-flow, expected to be less favorable for the parasite, significant group-level differences were found, albeit smaller than under common garden testing. The results show that the potential for group-level protection is lower than suggested by within-group genetic variation in lice count, and more so for environments giving the parasites easy access to hosts. These results cast some doubt about the efficacy of selective breeding for reduced lice count as a tool for group-level parasite control in densely populated fish farm environments. • Distribution of sea lice over salmon hosts within an environment is heritable. • The goal of selective breeding is to reduce overall parasite infestation success. • Divergently selected groups of salmon were challenge-tested in separate tanks. • No significant genetic group-protection effect on infestation in conditions favorable for the parasite. • Moderate but significant genetic group-protection effect in less favorable conditions for the parasite. [ABSTRACT FROM AUTHOR]

In view of zero-sum differential games, this paper addresses the fault-tolerant tracking control (FTTC) problem for nonlinear systems with multiplicative actuator faults. By augmenting the nominal system, the trajectory tracking problem is transformed into an optimal regulation problem. A fault observer is designed to estimate the multiplicative actuator fault factor. In order to better reflect system performance, a comprehensive performance index function including the trajectory tracking error, the control input, and the multiplicative actuator fault is constructed. By considering the control input and the multiplicative actuator fault as two players in the zero-sum differential game (ZSDG), the FTTC problem is thus transformed into a ZSDG problem. Then, a fault-tolerant control based on ZSDG is designed by solving the Hamilton–Jacobi-Isaacs (HJI) equation. Since the HJI equation is difficult to solve, a critic neural network is constructed to obtain its approximation, which constitutes the Nash equilibrium of the ZSDG. The closed-loop faulty system is guaranteed to be uniformly ultimately bounded via the Lyapunov's direct method. The effectiveness of the developed FTTC method is demonstrated by two simulation examples. [ABSTRACT FROM AUTHOR]

In this paper, we consider a security-aware and learning-based actuator selection formulation for cyber–physical systems. A set of actuators is chosen to be used by the system so that a metric of controllability and actuation attack resilience is maximized, given only partial knowledge of the physics of the system. Such a metric is related to the trace of a game theoretic Gramian, which is obtained by solving a zero-sum game between a defender who acts as a minimizer and wants to regulate the system, and an attacker who acts as a maximizer with an eventual goal to disrupt the regulation. To account for the lack of full knowledge of the system dynamics, we design an estimator that can learn the trace of the proposed game theoretic Gramian online, by utilizing data obtained across the system's trajectories. Subsequently, we propose a scheduling algorithm that utilizes the estimator to select actuators for the system in an online manner. In fact, we prove that the set of actuators that optimize the game-based metric will be learnt and scheduled permanently in finite time. Finally, we show that the learning-based actuator placement procedure can be utilized, in its steady state, to detect a large class of actuation attacks in a partially model-free manner. We carry out a simulation on an aircraft model to show the efficacy of our framework. [ABSTRACT FROM AUTHOR]

In this tutorial we study a safety analog of the classical zero-sum differential game with positive definite penalties on the state and the two inputs. Consider a nonlinear system affine in two inputs, which are called "offender" and "defender." Let the inputs have the opposing objectives in relation to an infinite-time cost which, in addition to penalizing the inputs of both agents, incorporates a safety index of the system (a barrier function), with the defender aiming to maximize the system safety and the offender aiming to minimize it. If there is a pair of (offender, defender) non-Nash feedback policies of the L g h form with a safe outcome, namely, where the defender maintains safety while the offender fails to violate safety, then there exists an inverse optimal pair of policies that attain a Nash equilibrium relative to the safety minimax objective. In the tutorial we study both deterministic and stochastic offenders. The deterministic offender applies its feedback through its deterministic input value, while the stochastic offender applies its feedback through its incremental covariance. In addition to Nash policies for a minimax offender–defender formulation, we provide feedback laws for the defender, in the scenario where the offender action is unrestricted by optimality, and where the defender ensures input-to-state safety in the deterministic and stochastic senses. This tutorial is derived from our recent article on inverse optimal safety filters, by setting the nominal control to zero and declaring the disturbance to be the offender agent. Among several illustrative examples, one is particularly interesting and unconventional. We consider a safety game played on a unicycle vehicle between its two inputs: the angular velocity and the linear velocity, as the opposing players. We consider two scenarios. In the first, the angular velocity, acting as an offender, attempts to run the vehicle into an obstacle by steering, while the linear velocity, acting as a defender, drives the vehicle forward or in reverse to prevent the vehicle being run into the obstacle. In the second scenario, the linear velocity acts as an offender and angular velocity acts as a defender (in the deterministic case by varying the heading rate; in the stochastic case by varying the variance of a white noise driving the heading rate). A "wind" towards the obstacle advantages the offender in both scenarios. The input policies derived are optimal in the sense of their opposite objectives, under the best possible policy of the opponent, under meaningful costs on their actions. The linear velocity input prevails, whether acting in the role of a defender, in which case the collision with the obstacle is prevented, or in the role of an offender, in which case the collision with the obstacle is achieved. [ABSTRACT FROM AUTHOR]

Recent work in the field of game theoretic modeling of natural language quantifiers introduces randomizing game rules within the setting of Giles's game for Łukasiewicz logic [3,15,16]. The randomization comes about in the form of a non-strategic player, called Nature. This can, taken as a mere selection principle, lead to an immense increase of expressivity, compared to the usual two player zero sum games of perfect information, like Hintikka's game or Giles's game [3]. We will define several games with randomizing game rules and investigate their expressivity, ending with a game which we call the NRG -game, with corresponding logic Ł α (Π). We give representation theorems for the class of semi-fuzzy deliberate choice quantifiers, and, as the main result of the paper, show how one can define the operations of Gödel and Product logic, and furthermore, those of all fuzzy logics, that are based on a continuous t-norm that is representable as a finite ordinal sum of the t-norms corresponding to Łukasiewicz, Gödel and Product logic [4] , within Ł α (Π) , as long as the domains of the interpretations are finite. [ABSTRACT FROM AUTHOR]

• Investigate a novel TrFBWM to yield the TrF criteria weights. • Develop the TrFZSG to rank the alternatives. • Propose a TrF-BWM-ZSG by fusinging the BWM and ZSG. This paper investigates a hybrid emergency decision-making (EDM) method called TrF-BWM-ZSG by combining the BWM (best-worst method) and ZSG (zero-sum game) in trapezoidal fuzzy (TrF) environment. Firstly, this paper defines the entropy and similarity for trapezoidal fuzzy numbers (TrFNs). Based on the entropy and similarity measures, a bi-objective programming model is constructed to distribute the weights for experts. Then, based on the multiplicative consistent trapezoidal fuzzy reference comparisons (TrFRCs), a novel TrFBWM is investigated to yield the TrF criteria weights. This novel TrFBWM highlights two innovations: (i) A crisp goal programming weight-determining model is constructed by using the multiplicative consistent TrFRCs; (ii) A reasonable consistency ratio is defined, in which the consistency index is determined according to the consistent transitivity of TrFRCs. Afterwards, the TrFZSG is developed to achieve the alternatives' ranking order by extending the classical ZSG into TrF environment. Lastly, the availability of the TrF-BWM-ZSG is corroborated through a real EDM case of Henan rainstorm disaster. The advantages of the TrF-BWM-ZSG are manifested by several valuable comparisons. Some desirable results derived by the TrF-BWM-ZSG are received: (i) The optimal alternative selected by the proposed TrF-BWM-ZSG suits the actual rescue situation; (ii) The proposed TrF-BWM-ZSG owns high generality since TrFN can reduce to crisp number, interval and triangular fuzzy number; (iii) The developed novel TrFBWM is helpful to preserve more inherent information of TrFNs. [ABSTRACT FROM AUTHOR]

The lack of effective defense resource allocation strategies and reliable multi-agent collaboration mechanisms lead to the low stability of Deep Reinforcement Learning (DRL)-based security defense strategies in several Internet of Things (IoT) applications. To address the aforementioned issues and approach real-world scenarios, we construct a grid-based adversarial security scenario, propose a two-stage zero-sum security game model (including the resource allocation stage and the patrolling detection stage), and design a t wo-stage s ecurity g ame s olution algorithm based on DRL for this game model, named TSGS. In the resource allocation stage, TSGS uses auxiliary action embedding and gradient approximation approach to compute the Nash Equilibrium (NE) allocation strategy, which addresses the problem of unreasonable allocation. In the patrolling detection stage, TSGS achieves team collaboration by training a multi-agent Dueling Deep-Q Network under the centralized training and decentralized execution (CTDE) framework, which solves the cooperation problem among multiple defense agents. In addition, we design and implement a multi-parameterized attacker model to make the attacker's behaviors more realistic in the game. Finally, the validity of the TSGS is verified by detailed experimental results for several adversarial experimental scenarios. Compared with the baseline methods, the defense strategy learned by TSGS has higher utility and greater robustness. Especially, TSGS achieves efficient collaboration during the patrolling detection stage after resource allocation by making full use of real-time information and communication. [ABSTRACT FROM AUTHOR]

We study the behavior of a stochastic variant of replicator dynamics in two-agent zero-sum games. Inspired by the well studied deterministic replicator equation, this model gives arguably a more realistic description of real world settings and, as we demonstrate here, exhibits totally distinct behavior when compared to its deterministic counterpart which is known to be recurrent. In more detail, we characterize the statistics of such systems by their invariant measures which can be shown to be entirely supported on the boundary of the space of mixed strategies. Depending on the noise strength we can furthermore characterize these invariant measures by finding accumulation of mass at specific parts of the boundary. In particular, regardless of the magnitude of noise, we show that any invariant probability measure is a convex combination of Dirac measures on pure strategy profiles, which correspond to vertices/corners of the agents' simplices. Thus, in the presence of stochastic perturbations, even in the most classic zero-sum settings, such as Matching Pennies, we observe a stark disagreement between the axiomatic prediction of Nash equilibrium and the evolutionary emergent behavior derived by an assumption of stochastically adaptive, learning agents. • Relevant model of stochastic replicator dynamics in two-agent zero-sum games. • Characterization of game dynamics via Nash equilibria and anti-equilibria. • Classification of noise-dependent behavior via attracting invariant measures. • Stark contrast to deterministic counterpart in terms of concentration on the boundary. • Exemplification of results for games with and without interior equilibria. [ABSTRACT FROM AUTHOR]

In this work, a new tracking control scheme is developed for underactuated autonomous underwater vehicle(UAUV) with unknown disturbance. First, an error tracking system is constructed using backstepping method, which transforms the tracking control problem into the two-player zero-sum game problem. One player is the optimal controller which is designed to make the UAUV track the desired trajectory. The other player is the unknown disturbance that represent the overall damping effect of the UAUV. Then, the single critic network based online policy iteration algorithm is designed to get the optimal control law and the worst-case disturbance, which can achieve the near-optimal control performance and relax the requirements for initial admissible control conditions. In order to improve the converge velocity of tracking error, the weight update law are designed. Furthermore, a discount coefficient is introduced into the performance index due to the nonlinearity and the complexity of UAUV. In addition, the stability of the UAUV is analyzed based on the Lyapunov stability theory the system is guaranteed to be uniformly ultimately bounded (UUB). Finally, the effectiveness of the proposed method is demonstrated through the real-time simulations on the UAUV model. • The tracking control problem is solved for UAUV with unknown disturbances based on an online zero-sum game theory proposed. • Only one critic network is employed to reduce the computation time and convergence time. • An online policy iteration algorithm and weights update law are designed to solve the online zero-sum game problem. • A discount coefficient is introduced into the performance index due to the nonlinearity and complexity of UAUV. [ABSTRACT FROM AUTHOR]

Since the multi-time scales and strong nonlinearity are objectively existed in the practical industrial processes, it is extremely challenging to design control policy for optimizing the industrial operation. Therefore, this paper focuses on the multi-time scale optimal operational control (OOC) problem with unmodeled dynamics. Under the framework of reinforcement learning (RL) technology, a self-learning model-free method by integrating zero-sum game and singular perturbation (SP) theories is proposed in this paper, such that composite controllers with signal compensation can be found under which the suboptimal operation of the whole process can be achieved using only measured data. The remarkable highlight in this paper is the novel design method of the composite controllers with signal compensations for unknown nonlinear systems with multi-rate operations. Finally, the effectiveness of the proposed method is verified by a practical example and a numerical example. [ABSTRACT FROM AUTHOR]

• The fault-tolerant tracking control problem is converted into a zero-sum game problem. • The Bellman equation and the Riccati equation can be established. • An off-policy Q-learning algorithm is obtained. Traditional model-based control methods are often not applicable in industrial processes given the typical situation that model parameters are unknown, coefficient matrices are difficult to obtain, and system states are unpredictable. Accordingly, an output feedback fault-tolerant control method based on zero-sum game theory and off-policy Q-learning is presented in this study, with the aim of achieving smooth operation and good tracking performance for industrial processes that often contain sensor faults and disturbances. The specific steps are as follows. First, a system tracking error is introduced into the system to realize a novel extended model. Second, by establishing a performance index function and combining it with minimax theory, the fault-tolerant tracking control problem is converted into a zero-sum game problem. The Bellman and Riccati equations can be established after analyzing the relationship between the performance index and value functions. Then, the Q-function is introduced, and an off-policy Q-learning algorithm is combined with the Kronecker product without knowledge of system model parameters to design an optimal controller unbiased to detection noise. Finally, the effectiveness of the algorithm is verified by considering the injection molding process as an example. The experimental results validate that the designed controller demonstrates good control and extends the range of tolerable faults while maintaining good tracking performance. [ABSTRACT FROM AUTHOR]

Conflicts among various spatial planning policies have created a spatial game over land function and utilization due to the limitation of land resources. We quantitatively assess the evolutionary game process among three functional space types (agricultural production, urban–rural living, and blue–green ecological) using the concept of game theory during the period of 1980–2020 in Henan Province, China, as a case study. A research framework for assessing the spatial game process was established, and a measurement method based on spatial game type (SGT) and spatial game intensity (SGI) is proposed. The results demonstrate that the evolutionary game process among three functional space types exhibited significant spatiotemporal variation and spatial agglomeration. SGI increased from low to high values and then decreased again, and exhibited spatial autocorrelation. SGT supported classification into positive-sum and zero-sum games. The period of 1990–2000 had the largest SGI and the most counties with high-intensity zero-sum games. A highly urbanized urban center showed a high-intensity zero-sum game at the expense of agricultural production spaces. Balancing the areas devoted to various uses, enhancing spatial connectivity and improving functional mixing degree of land use may be effective ways to alleviate noncooperative games and spatial conflicts among these three functional space types. • A research framework and algorithm for the quantification of the evolutionary game process among three functional space types were proposed. • Spatial game intensity (SGI) and spatial game type (SGT) were introduced to reflect the dynamics of spatial game process. • Both SGI and SGT exhibited distinct spatiotemporal heterogeneity and spatial agglomeration. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel deep learning approach for predicting saddle points in stochastic two-player zero-sum games. Our method combines neurodynamic optimization and deep neural networks. First, we model the stochastic two-player zero-sum game as an ordinary differential equation (ODE) system using neurodynamic optimization. Second, we develop a neural network to approximate the solution to the ODE system, which includes the saddle point prediction for the game problem. Third, we introduce a specialized algorithm for training the neural network to enhance the accuracy of the saddle point prediction. Our experiments demonstrate that our model outperforms existing approaches, yielding faster convergence and more accurate saddle point predictions. [ABSTRACT FROM AUTHOR]

This paper investigates the fuzzy approximation-based optimal consensus control problem for nonlinear multiagent systems with unknown perturbations. By constructing local error dynamics, the considered optimal consensus problem is reformulated as finding Nash-equilibrium solutions to zero-sum games. Then, by using sliding mode control technology and the concept of hierarchical design, a series of control signals are sequentially designed to regulate the consensus error and minimize the local value function. In addition, an identifier-critic architecture is developed by using generalized fuzzy hyperbolic models, where the identifier is employed to relax the requirement for complete system dynamics information, and the hierarchical sliding mode surface-based critic network is applied to approximate optimal control inputs. Finally, A simulation example is presented to illustrate the validity of the proposed approach. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of fault tolerant control (FTC) for reentry vehicles (RV) subject to actuator fault and perturbation. In contrast to the majority of papers assuming bounded and exogenous actuator fault effect, the adaptive dynamic programming (ADP) control algorithm has been considered here to tackle with the dependence of closed-loop states induced by the setup of an active FTC strategy. By this way, the FTC problem formulation takes theoretically into account the fact that an actuator fault can destabilize a closed-loop. First, the FTC problem in attitude tracking is transformed into an error-optimal control problem. Next, a cost function based on zero-sum game theory is considered to achieve tracking under the joint influence of state-dependent actuator faults and perturbations. Lyapunov theory has been next used to prove the stability of FTC architecture and the network weight convergence. Benefits of proposed ADP-FTC algorithm is finally highlighted on a numerical benchmark of RV. [ABSTRACT FROM AUTHOR]

Stable operations of control systems play a vital role in mission accomplishments of the second-order nonlinear systems, such as six-axis robots used in intelligent production lines, industrial equipment and control systems. In this paper, a fault-tolerant control scheme is developed for a class of second-order nonlinear control system under actuator bias faults and loss of effectiveness faults via the zero-sum differential game method. Based on the backstepping method, a controller is designed to ensure system tracking performance. Then by the zero-sum differential game method, a fault-tolerant controller is designed for the equivalent error system. Simulation results show the validity of the designed fault-tolerant control scheme. • The stability of the system and the optimality of control consumption are achieved. • Both unknown bias and loss of effectiveness faults are solved. • Uncertainty of the control gain matrix caused by the fault pattern matrix is solved. [ABSTRACT FROM AUTHOR]

This paper aims to investigate an innovative framework to handle emergency response scheme selection (ERSS) issues by integrating TODIM and TPZSG (two-person zero-sum game) methods under novel T-spherical hesitant probabilistic fuzzy set (T-SHPFS) environments. First, T-SHPFS is defined as an extension of the existing tools, which can depict the complex assessment information including several possible values of the various membership functions' degrees and the associated statistical uncertainty information. Concomitantly, T-SHPFS's normalization method, comparison laws, operation rules, cross-entropy measure and Hausdorff distance are explored. Then, an objective attribute weight determining model is constructed, considering the credibility of T-SHPF evaluations and the divergence degrees between attribute assessments simultaneously. Next, an integrated TODIM-TPZSG decision-making approach is developed to select the most desirable emergency response scheme. Finally, an illustrative example concerning the selection of the best medical waste disposal method during the COVID-19 epidemic is conducted to verify the effectiveness of the proposed TODIM-TPZSG method. Sensitivity analysis and comparisons between the TODIM-TPZSG and other representative methods are also provided to demonstrate the superiorities of the proposed method. The results reveal that the developed T-SHPFSs give DMs more assessment freedom; the proposed TODIM-TPZSG approach considers the decision makers' psychological behaviors; the ranking results of the proposed method can reflect the specific divergence degrees among the alternatives; and the needed computation burden and computational complexity are low and less affected by the number of alternatives and criteria than most current ERSS methods. • Proposing T-spherical hesitant probabilistic fuzzy set (T-SHPFS), a novel fuzzy set, to describe intricate information, which renders experts more freedom in evaluation processes. • Establishing the normalization method, comparison laws, operation rules, cross-entropy measure and Hausdorff distance of T-SHPFSs. • Constructing an objective attribute weight determining model, considering the credibility of T-SHPF evaluations and the divergence degree between attribute assessments simultaneously. • Devising an integrated TODIM-TPZSG decision-making approach for selecting the most desirable emergency response scheme. [ABSTRACT FROM AUTHOR]

This work presents an output-feedback policy learning algorithm underlining input–output system data for distributed robust optimal synchronization of heterogeneous multi-agent systems. The output-feedback synchronization problem in the context of this work is formulated via robust output regulation and reinforcement learning modeling the interactions among agents by a zero-sum game. The proposed learning and control structure only requires the local system data for each agent and distributed output data among communicating neighbors. We utilize system-level synchysis for the continuous-time state reconstruction for the distributed learning with convergence and stability proofs under the proposed output-feedback policy for solving the zero-sum game. We further show that policy learning is assured under the proposed data criteria relating to input–output data only rather than any inter-immediate gains from policy iterations. Based on the cooperative robust output regulation, this work gains robustness after the learning is complete and establishes an output data-driven distributed optimal robust synchronization without knowing accurate system dynamics. A numerical example shows the effectiveness of the proposed learning algorithm. [ABSTRACT FROM AUTHOR]

Abstract We present a novel approach to solve and create a two person zero sum matrix game by using matrix norms. Especially, we show how to obtain approximated game value for any zero sum matrix game without solving any equations using our approaches. We firstly, give the results of the lemmas for the game value depend on the matrix norms of the payoff matrix and some constants k containing the game value v. Then, we introduce row-wise and column-wise induced matrix for the payoff matrix. Moreover, we improve our approaches and present some new theorems for the game value to obtain some inequalities which depend on only the 1 − n o r m and ∞ − n o r m of the payoff matrix. Furthermore, we state the min–max theorem for p max and p min which are the maximum and minimum elements of the mixed strategy set, respectively. Finally, we illustrate and show the consistency of our approaches with some test examples. To the best of our knowledge, this is the first study in the literature that is used the matrix norms in game theory. [ABSTRACT FROM AUTHOR]

This paper considers a new class of deterministic finite-time horizon, two-player, zero-sum differential games (DGs) in which the maximizing player is allowed to take continuous and impulse controls whereas the minimizing player is allowed to take impulse control only. We seek to approximate the value function, and to provide a verification theorem for this class of DGs. By means of dynamic programming principle (DPP) in viscosity solution (VS) framework, we first characterize the value function as the unique VS to the related Hamilton–Jacobi-Bellman–Isaacs (HJBI) double-obstacle equation. Next, we prove that an approximate value function exists, that it is the unique solution to an approximate HJBI double-obstacle equation, and converges locally uniformly towards the value function of each player when the time discretization step goes to zero. Moreover, we provide a verification theorem which characterizes a Nash-equilibrium (NE) for the DG control problem considered. Finally, by applying our results, we derive a new continuous-time portfolio optimization model, and we provide related computational algorithms and numerical results. [ABSTRACT FROM AUTHOR]

X-ray computed tomography (CT) and positron emission tomography (PET) are two of the most commonly used medical imaging technologies for the evaluation of many diseases. Full-dose imaging for CT and PET ensures the image quality but usually raises concerns about the potential health risks of radiation exposure. The contradiction between reducing the radiation exposure and remaining diagnostic performance can be addressed effectively by reconstructing the low-dose CT (L-CT) and low-dose PET (L-PET) images to the same high-quality ones as full-dose (F-CT and F-PET). In this paper, we propose an Attention–encoding Integrated Generative Adversarial Network (AIGAN) to achieve efficient and universal full-dose reconstruction for L-CT and L-PET images. AIGAN consists of three modules: the cascade generator, the dual-scale discriminator and the multi-scale spatial fusion module (MSFM). A sequence of consecutive L-CT (L-PET) slices is first fed into the cascade generator that integrates with a generation-encoding-generation pipeline. The generator plays the zero-sum game with the dual-scale discriminator for two stages: the coarse and fine stages. In both stages, the generator generates the estimated F-CT (F-PET) images as like the original F-CT (F-PET) images as possible. After the fine stage, the estimated fine full-dose images are then fed into the MSFM, which fully explores the inter- and intra-slice structural information, to output the final generated full-dose images. Experimental results show that the proposed AIGAN achieves the state-of-the-art performances on commonly used metrics and satisfies the reconstruction needs for clinical standards. • A generative adversarial network AIGAN is proposed for low-dose CT and PET image reconstruction. • The cascade generator allows the low-dose images to be enhanced twice with attention-encoding. • The dual-scale discriminator plays the zero-sum game with the cascade generator for two rounds. • The multi-scale spatial fusion module exploits inter- and intra-slice structural information. • Clinical reading and cross-domain generalization validation demonstrate the potential of AIGAN. [ABSTRACT FROM AUTHOR]

Quantitative 'completion criteria' agreed between mining companies and regulatory authorities are typically used to assess restoration outcomes. Completion criteria provide the basis for determining whether mining companies have fully met their obligations for site restoration. Relationships among potentially conflicting criteria, particularly around site productivity (e.g., timber production), the return of biodiversity, long-term ecosystem trajectories and other ecosystem services remain poorly understood during the development of mine restoration. For the jarrah forest of southwest Australia, we manipulated site productivity in post bauxite mine restoration by establishing four initial Eucalyptus marginata (jarrah) tree stocking rates in factorial combination with seeding a range of large N 2 -fixing understorey legume species. We measured jarrah growth rates over the 32-year duration of the experiment, soil-N concentrations at 3 and 32 years-of-age and understorey plant species richness and cover at 15 and 32 years-of-age. Stand basal area over bark (BA) increased with increasing tree stocking rates. However, there was a significant negative effect of increasing stand BA on understorey plant cover and species richness. These negative effects were still significant 32 years after the commencement of the experiment. Seeding large understorey legumes increased total soil-N. However, this did not increase tree growth. Rather, seeding large legumes reduced tree survival and diameter at breast height, and reduced understorey species richness, even though the applied seed mix in the large legume treatment contained 14 more species compared with the control. Our data highlight trade-offs between individual aspects of restoration success, such as tree stocking rates and understorey species richness. Further these results suggest reducing site productivity, by the judicious use of large understorey legumes and establishing low tree stocking rates, may facilitate the return of diverse vegetation communities. Finally, it is important to recognise that when restoration success is viewed through the lens of one-dimensional metrics such as completion criteria, restoration may function as a 'zero-sum game' in which a gain for one outcome (e.g., N 2 -fixation) may result in a loss to one or more other outcomes (e.g., tree growth and species richness). [ABSTRACT FROM AUTHOR]

In this paper, a data-driven approximate neural network (NN) learning scheme is developed to solve the multi-player nonzero-sum (NZS) game problem with completely unknown system dynamics. An augmented NN identifier based on a new parameter estimation algorithm is first established to approximate the completely unknown system dynamics. Then approximated dynamic programming (ADP) with neural networks is constructed to approximate the optimal solutions of the coupled Hamilton-Jacobi equations for each player. The approximated NN value functions are then used to synchronously calculate the optimal control policies for every player. The identifier and ADP NN weights are online updated with the system input-output data based on a novel adaptive law, which could achieve a faster convergence speed. Moreover, the convergence of all NN weights and the stability of the closed-loop system are proved based on the Lyapunov approach. Finally, a dual-driven servo motor system and a three-player nonlinear game system are simulated to verify the feasibility of the developed methods. [ABSTRACT FROM AUTHOR]

Studies of ethno-nationalist conflict have repeatedly underlined the significance of policy interventions that seek to de-territorialise contested space after armed conflict and create more plural societies. Creating ‘shared’ space in divided societies is often critically important and inextricably linked to peacebuilding. However much of this scholarship has tended to focus on the relative success or failure of such policies. This paper conversely explores the ‘unintended consequences’ (Merton, 1936) of legislating around fragile public space in Northern Ireland and considers its potential to undermine, rather than reinforce efforts to transition to peace. Drawing on a body of work around unintended consequences, territorial socialisation and peacebuilding, we argue that such legislation in ethno-nationalist societies emerging from conflict is a double-edged sword which can be utilised both explicitly and implicitly to reactivate tribal spatial politics and exacerbate divisions in deeply divided societies. [ABSTRACT FROM AUTHOR]