Search

A generalized class of the discrete game of timing is solved as a finite zero-sum game defined on a symmetric lattice of the unit square. The game is a progressive discrete silent duel whose kernel is skew-symmetric, and the players, referred to as duelists, have identical shooting accuracy functions featured with an accuracy proportionality factor a and a power constant β describing nonlinearity of the shooting accuracy. As the duel starts, time moments of possible shooting become denser by a geometric progression. Apart from the duel beginning and end time moments, every following time moment is the partial sum of the respective geometric series. Due to the skew-symmetry, both the duelists have the same optimal strategies and the game optimal value is 0. As the duelist has a single bullet, there is no reason for considering solutions in mixed strategies, if any, with non-singleton supports. If a ⩾ 2 β − 1 , the duelist's optimal strategy is the middle of the duel time span; otherwise, the duel solution may be not a pure strategy solution. The case of quadratic accuracy for β = 2 is further considered, which is reasoned by that linearly developing real-time processes are quite rare, and the quadratic accuracy is a non-linearity pattern that makes an interaction model more reliable by slightly diminishing the duelist's confidence. Thus, the case of a < 2 β − 1 is thoroughly studied for pure strategy solutions. A boundary case of a is found, by which the duel has four pure strategy solutions which are of the time moment preceding the duel end moment and the duel end moment itself. • Discrete game of timing is solved as a finite zero-sum game on a symmetric lattice. • Duelist is featured with shooting accuracy proportionality and nonlinearity factors. • The breaking point of the accuracy is found above which pure strategy solutions exist. • Pure strategy solutions are found below the breaking point for the quadratic accuracy. [ABSTRACT FROM AUTHOR]

The potential emergence of artificial general intelligence (AGI) systems has sparked intense debate among researchers, policymakers, and the public due to their potential to surpass human intelligence in all domains. This note argues that for an AI to be considered "general", it should achieve superhuman performance not only in zero-sum games but also in general-sum games, where winning or losing is not clearly defined. In this note, I propose a game-theoretic framework that captures the strategic interactions between a representative human agent and a potential superhuman machine agent. Four assumptions underpin this framework: Superhuman Machine, Machine Strategy, Rationality, and Strategic Unpredictability. The main result is an impossibility theorem, establishing that these assumptions are inconsistent when taken together, but relaxing any one of them results in a consistent set of assumptions. This note contributes to a better understanding of the theoretical context that can shape the development of superhuman AI. • A game between a human vs an Artificial General Intelligence system is introduced. • An AI system is general if it is successful in both zero-sum and general-sum games. • Assumptions are Superhuman AI, Machine Strategy, Rationality, and Unpredictability. • An impossibility theorem shows that these assumptions lead to inconsistency. • The result is tight; relaxing any of the assumptions resolves the inconsistency. [ABSTRACT FROM AUTHOR]

The well-known H ∞ control design gives robustness to a controller by rejecting perturbations from the external environment, which is difficult to do for completely unknown affine nonlinear systems. Accordingly, the immediate objective of this paper is to develop an on-line real-time synergetic learning algorithm, so that a data-driven H ∞ controller can be received. By converting the H ∞ control problem into a two-player zero-sum game, a model-free Hamilton–Jacobi–Isaacs equation (MF-HJIE) is first derived using off-policy reinforcement learning, followed by a proof of equivalence between the MF-HJIE and the conventional HJIE. Next, by applying the temporal difference to the MF-HJIE, a synergetic evolutionary rule with experience replay is designed to learn the optimal value function, the optimal control, and the worst perturbation, that can be performed on-line and in real-time along the system state trajectory. It is proven that the synergistic learning system constructed by the system plant and the evolutionary rule is uniformly ultimately bounded. Finally, simulation results on an F16 aircraft system and a nonlinear system back up the tractability of the proposed method. • A model-free Hamiltonian–Jacobi–Isaacs equation is derived for H ∞ control problems. • A real-time on-line evolutionary rule for tuning neural network weights is developed. • The system and neural network approximation errors achieve synergetic learning. [ABSTRACT FROM AUTHOR]

In this paper, a novel parallel learning framework is developed to solve zero-sum games for discrete-time nonlinear systems. Briefly, the purpose of this study is to determine a tentative function according to the prior knowledge of the value iteration (VI) algorithm. The learning process of the parallel controllers can be guided by the tentative function. That is to say, the neighborhood of the optimal cost function can be compressed within a small range via two typical exploration policies. Based on the parallel learning framework, a novel dichotomy VI algorithm is established to accelerate the learning speed. It is shown that the parallel controllers will converge to the optimal policy from contrary initial policies. Finally, two typical systems are used to demonstrate the learning performance of the constructed dichotomy VI algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we present two-timescale neurodynamic optimization approaches to distributed minimax optimization. We propose four multilayer recurrent neural networks for solving four different types of generally nonlinear convex–concave minimax problems subject to linear equality and nonlinear inequality constraints. We derive sufficient conditions to guarantee the stability and optimality of the neural networks. We demonstrate the viability and efficiency of the proposed neural networks in two specific paradigms for Nash-equilibrium seeking in a zero-sum game and distributed constrained nonlinear optimization. [ABSTRACT FROM AUTHOR]

In this paper, a novel adaptive critic control method is designed to solve an optimal H ∞ tracking control problem for continuous nonlinear systems with nonzero equilibrium based on adaptive dynamic programming (ADP). To guarantee the finiteness of a cost function, traditional methods generally assume that the controlled system has a zero equilibrium point, which is not true in practical systems. In order to overcome such obstacle and realize H ∞ optimal tracking control, this paper proposes a novel cost function design with respect to disturbance, tracking error and the derivative of tracking error. Based on the designed cost function, the H ∞ control problem is formulated as two-player zero-sum differential games, and then a policy iteration (PI) algorithm is proposed to solve the corresponding Hamilton–Jacobi–Isaacs (HJI) equation. In order to obtain the online solution to the HJI equation, a single-critic neural network structure based on PI algorithm is established to learn the optimal control policy and the worst-case disturbance law. It is worth mentioning that the proposed adaptive critic control method can simplify the controller design process when the equilibrium of the systems is not zero. Finally, simulations are conducted to evaluate the tracking performance of the proposed control methods. [ABSTRACT FROM AUTHOR]

• The study explores changes in charitable giving after a major domestic disaster - Hurricane Sandy. • Total donations increase in the immediate aftermath of Hurricane Sandy. • Donations for Sandy relief are, on average, not substituted with similar projects. • Donations for other disaster recovery projects increase in the immediate aftermath of Hurricane Sandy. • In affected regions, these changes in charitable giving appear to be more pronounced than in regions further away. This paper re-opens the discussion of whether new demand for charitable aid generates additional giving or is ultimately a zero-sum game. I employ the exogenous variation in donations provided by Hurricane Sandy and a unique data set from a large crowdfunding platform to analyze giving patterns around the disaster. The study consists of two parts. The first part examines whether a major disaster like Hurricane Sandy increases the altruism budget and if and how donors substitute their donations to the victims of the disaster. The second part focuses on the disaster experience and whether the altruism budget and substitution differ with the proximity to the disaster and resulting damage. The findings indicate that the altruism budget increases in line with earlier studies and that not only victims of Hurricane Sandy, but also victims of other major disasters benefit from this increase at least in the immediate aftermath. The analyses provide no evidence for substitution with similar projects. Giving for victims of Hurricane Sandy and other disasters seems to be slightly more pronounced in affected regions relative to the control group of regions further away. [ABSTRACT FROM AUTHOR]

In this paper, we present a Q-learning framework for solving finite-horizon zero-sum game problems involving the H ∞ control of linear system without knowing the dynamics. Research in the past mainly focused on solving problems in infinite horizon with completely measurable state. However, in the practical engineering, the system state is not always directly accessible, and it is difficult to solve the time-varying Riccati equation associated with the finite-horizon setting directly either. The main contribution of the proposed model-free algorithm is to determine the optimal output feedback policies without measurement state in finite-horizon setting. To achieve this goal, we first describe the Q-function caused by finite-horizon problems in the context of state feedback, then we parameterize the Q-functions as input–output vectors functions. Finally, the numerical examples on aircraft dynamics demonstrate the algorithm's efficiency. [ABSTRACT FROM AUTHOR]

We study zero-sum stochastic games for controlled discrete time Markov chains with risk-sensitive average cost criterion with countable/compact state space and Borel action spaces. The payoff function is nonnegative and possibly unbounded for countable state space case and for compact state space case it is a real-valued and bounded function. For countable state space case, under a certain Lyapunov type stability assumption on the dynamics we establish the existence of the value and a saddle point equilibrium. For compact state space case we establish these results without any Lyapunov type stability assumptions. Using the stochastic representation of the principal eigenfunction of the associated optimality equation, we completely characterize all possible saddle point strategies in the class of stationary Markov strategies. Also, we present and analyze an illustrative example. [ABSTRACT FROM AUTHOR]

In this paper, a synergetic learning structure-based neuro-optimal fault tolerant control (SLSNOFTC) method is proposed for unknown nonlinear continuous-time systems with actuator failures. Under the framework of the synergetic learning structure (SLS), the optimal control input and the actuator failure are viewed as two subsystems. Then, the fault tolerant control (FTC) problem can be regarded as a two-player zero-sum differential game according to the game theory. A radial basis function neural network-based identifier, which uses the measured input/output data, is constructed to identify the completely unknown system dynamics. To develop the SLSNOFTC method, the Hamilton–Jacobi–Isaacs equation is solved by an asymptotically stable critic neural network (ASCNN) which is composed of cooperative adaptive tuning laws. Besides, with the help of the Lyapunov stability analysis, the identification error, the weight error of ASCNN, and all signals of closed-loop system are guaranteed to be converged to zero asymptotically, rather than uniformly ultimately bounded. Numerical simulation examples further verify the effectiveness and reliability of the proposed method. [ABSTRACT FROM AUTHOR]

We investigate the interplay between forward uncertain difference equations and backward stochastic difference equations, which belong to distinct mathematical frameworks: uncertainty theory and probability theory, respectively. By establishing the connections between uncertain expectation, probabilistic expectation, and chance expectation, we develop the chance theory framework for analyzing uncertain stochastic hybrid zero-sum games constrained by both forward uncertain difference equations and backward stochastic difference equations. Through an equivalent transformation, we convert these games into deterministic difference equation solving problems. By backward-solving the difference equations and applying mathematical induction, analytical expressions for the equilibrium solutions of two types of uncertain stochastic hybrid zero-sum games are obtained. Furthermore, we explore solution methods for zero-sum games with different operational orders of uncertain and probabilistic expectations and highlight the fundamental differences through numerical examples. • Present uncertain stochastic hybrid zero-sum games of forward and backward systems. • Chance theory framework aids analysis of uncertain stochastic hybrid zero-sum games. • Transforms these games into deterministic difference equation solving problems. • Explores solution methods for games with different expectation operational orders. • Provides numerical examples to illustrate effectiveness of solution methods. [ABSTRACT FROM AUTHOR]

This paper investigates the optimal tracking issue for continuous-time (CT) nonlinear asymmetric constrained zero-sum games (ZSGs) by exploiting the neural critic technique. Initially, an improved algorithm is constructed to tackle the tracking control problem of nonlinear CT multiplayer ZSGs. Also, we give a novel nonquadratic function to settle the asymmetric constraints. One thing worth noting is that the method used in this paper to solve asymmetric constraints eliminates the strict restriction on the control matrix compared to the previous ones. Further, the optimal controls, the worst disturbances, and the tracking Hamilton–Jacobi–Isaacs equation are derived. Next, a single critic neural network is built to estimate the optimal cost function, thus obtaining the approximations of the optimal controls and the worst disturbances. The critic network weight is updated by the normalized steepest descent algorithm. Additionally, based on the Lyapunov method, the stability of the tracking error and the weight estimation error of the critic network is analyzed. In the end, two examples are offered to validate the theoretical results. [ABSTRACT FROM AUTHOR]

In this work, we consider systems whose components suffer from clock offsets and quantization and study the effect of those on a reinforcement learning (RL) algorithm. Specifically, we consider an off-policy iterative RL algorithm for continuous-time systems, which uses input and state data to approximate the Nash-equilibrium of a zero-sum game. However, the data used by this algorithm are not consistent with one another, in that each of them originates from a slightly different time instant of the past, hence putting the convergence of the algorithm in question. We prove that, given that these timing inconsistencies remain below a certain threshold, the iterative off-policy RL algorithm will still converge epsilon-closely to the desired Nash policy. However, this result is conditional to a certain Lipschitz continuity and differentiability condition on the input-state data collected, which is indispensable in the presence of clock offsets. A similar result is also derived when quantization of the measured state is considered. Finally, unlike prior work, we provide a sufficiently rich data condition for the execution of the iterative RL algorithm, which can be verified a priori across all iteration indices. Simulations are performed, which verify and clarify theoretical findings. [ABSTRACT FROM AUTHOR]

In this paper, an event-triggered tracking optimization control method is proposed for nonlinear uncertain systems with H ∞ discounted cost. First, the linearization method using the infinite series expansion theorem is given. And the tracking control design problem is solved by appropriate state augment. Secondly, the zero sum differential game strategy is introduced to address the control design problem of uncertain systems with event-triggered inputs. The actor-critic-disturbance networks framework is used to approximate the optimization of event- triggered controller strategy, optimization costs, and the disturbance strategy, respectively. Due to the consideration of discrete time and continuous time dynamics, we put the impulsive system method to use and choose appropriate Lyapunov functions to prove stability and boundedness, at the same time, there will be no infinite triggering situation. Finally, the effectiveness of the proposed scheme is demonstrated by an example of a practical tractor system with n trailers through simulation compared with another typical design method. [ABSTRACT FROM AUTHOR]

In this paper, an adjustable Q-learning scheme is developed to solve the discrete-time nonlinear zero-sum game problem, which can accelerate the convergence rate of the iterative Q-function sequence. First, the monotonicity and convergence of the iterative Q-function sequence are analyzed under some conditions. Moreover, by employing neural networks, the model-free tracking control problem can be overcome for zero-sum games. Second, two practical algorithms are designed to guarantee the convergence with accelerated learning. In one algorithm, an adjustable acceleration phase is added to the iteration process of Q-learning, which can be adaptively terminated with convergence guarantee. In another algorithm, a novel acceleration function is developed, which can adjust the relaxation factor to ensure the convergence. Finally, through a simulation example with the practical physical background, the fantastic performance of the developed algorithm is demonstrated with neural networks. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a non-linear Snell envelope which at each time represents the maximal value that can be achieved by stopping a BSDE with constrained jumps. We establish the existence of the Snell envelope by employing a penalization technique and the primary challenge we encounter is demonstrating the regularity of the limit for the scheme. Additionally, we relate the Snell envelope to a finite horizon, zero-sum stochastic differential game, where one player controls a path-dependent stochastic system by invoking impulses, while the opponent is given the opportunity to stop the game prematurely. Importantly, by developing new techniques within the realm of control randomization, we demonstrate that the value of the game exists and is precisely characterized by our non-linear Snell envelope. [ABSTRACT FROM AUTHOR]

This paper aims at solving a stochastic two-player zero-sum Nash game problem studied in Singh and Lisser (2019). The main contribution of our paper is that we model this game problem as a dynamical neural network (DNN for short). In this paper, we show that the saddle point of this game problem is the equilibrium point of the DNN model, and we study the globally asymptotically stable of the DNN model. In our numerical experiments, we present the time-continuous feature of the DNN model and compare it with the state-of-the-art convex solvers, i.e., Splitting conic solver (SCS for short) and Cvxopt. Our numerical results show that our DNN method has two advantages in dealing with this game problem. Firstly, the DNN model can converge to a better optimal point. Secondly, the DNN method can solve all problems, even when the problem size is large. [ABSTRACT FROM AUTHOR]

For industrial processes with external disturbance and actuator failure, off-policy reinforcement learning-based novel model-free minmax fault-tolerant control is proposed in this paper to solve H ∞ fault-tolerant tracking control problem. An augmented model equivalent to the original system is constructed, and the state of the new augmented model is composed of state increment and tracking error of the original system. The original H ∞ fault-tolerant tracking problem was transformed into the linear quadratic zero-sum game problem by establishing performance index function, and the Game Algebraic Riccati Equation (GARE) was established. Then Q function was introduced and the Off-policy reinforcement learning algorithm was designed. Different from the traditional model-based fault-tolerant control method, the proposed algorithm does not need the knowledge of system dynamics, and it can learn from the measured data of the system trajectory to solve the GARE. In addition, it is proved that the probing noise added to satisfy the persistent excitation condition does not cause bias. A simulation example of injection molding process is used to verify the effectiveness of the proposed algorithm. • A new off-policy reinforcement learning algorithm based novel model-free minmax fault-tolerant control is proposed. • The optimal control strategy is transformed into minmax zero-sum game method. • Off-policy Q-learning algorithm is used to solve the optimal control strategy and the worst external disturbance. [ABSTRACT FROM AUTHOR]

Positron emission tomography (PET) imaging is widely used in medical imaging for analyzing neurological disorders and related brain diseases. Usually, full-dose imaging for PET ensures image quality but raises concerns about potential health risks of radiation exposure. The contradiction between reducing radiation exposure and maintaining diagnostic performance can be effectively addressed by reconstructing low-dose PET (L-PET) images to the same high-quality as full-dose (F-PET). This paper introduces the Multi Pareto Generative Adversarial Network (MPGAN) to achieve 3D end-to-end denoising for the L-PET images of human brain. MPGAN consists of two key modules: the diffused multi-round cascade generator (G D m c) and the dynamic Pareto-efficient discriminator (D P e d), both of which play a zero-sum game for n (n ∈ 1 , 2 , 3) rounds to ensure the quality of synthesized F-PET images. The Pareto-efficient dynamic discrimination process is introduced in D P e d to adaptively adjust the weights of sub-discriminators for improved discrimination output. We validated the performance of MPGAN using three datasets, including two independent datasets and one mixed dataset, and compared it with 12 recent competing models. Experimental results indicate that the proposed MPGAN provides an effective solution for 3D end-to-end denoising of L-PET images of the human brain, which meets clinical standards and achieves state-of-the-art performance on commonly used metrics. • We propose a multi-round cascade generator with noise injection for adaptive PET image denoising. • A dynamic Pareto-efficient discriminator enhances PET image differentiation with the generator. • We present a 3D sliding-patch strategy for efficient PET denoising without multi-GPU use. • Experimental results show our approach outperforms leading methods in PET image denoising. [ABSTRACT FROM AUTHOR]

From a perspective of designing or engineering for opinion formation games in social networks, the opinion maximization (or minimization) problem has been studied mainly for designing seeding algorithms that aim at selecting a subset of nodes to control their opinions. We first define a two-player zero-sum Stackelberg game of competitive opinion optimization by letting the player under study as the leader minimize the sum of expressed opinions by doing so-called "internal opinion design", knowing that the other adversarial player as the follower is to maximize the same objective by also conducting her own internal opinion design. We furthermore consider multiagent learning, specifically using the Optimistic Gradient Descent Ascent, and analyze its convergence to equilibria in the simultaneous-game version of competitive opinion optimization. [ABSTRACT FROM AUTHOR]

In this paper, we mainly consider the solution of two-person zero-sum fuzzy matrix games with payoffs of triangular fuzzy numbers. Contrary to the literature, we focus on developing the methods to solve the game directly based on only the norms of the payoff matrix holistically, without solving a linear programming problem or handling sub-games created by taking the components of fuzzy numbers separately. For this purpose, we first present fuzzy versions of 1-norm and ∞-norm with the help of a ranking function and develop the fuzzy matrix norm method to obtain an approximate solution of the zero-sum fuzzy matrix game. In addition to this approach, we provide the fuzzy extended matrix norm method as an enhanced version of the method, which involves the use of newly defined fuzzy matrix norms. Thus, we managed to avoid the complexity of the optimization process of the linear programming problem via the proposed matrix norm-based methods. Finally, we illustrate the implementation of the methods by considering several benchmark examples and a 3 × 3 fuzzy matrix game. • Alternative methods for the solution of the zero-sum fuzzy matrix game are introduced. • Fuzzy norms based on a ranking function are defined. • Holistic methods are proposed based on only the norms of the fuzzy payoff matrix. • The approximated fuzzy game value is obtained with new methods. • Comprehensive implementations of the methods are presented. [ABSTRACT FROM AUTHOR]

This paper studies a class of zero-sum stopping game in a regime switching model. A verification theorem as a sufficient criterion for Nash equilibriums is established based on a set of variational inequalities (VIs). Under an appropriate regularity condition for solutions to the VIs, a suitable system of algebraic equations is derived via the so-called smooth-fit principle. Explicit Nash equilibrium stopping rules of threshold-type for the two players and the corresponding value function of the game in closed form are obtained. Numerical experiments are reported to demonstrate the dependence of the threshold levels on various model parameters. A reduction to the case with no regime switching is also presented as a comparison. [ABSTRACT FROM AUTHOR]

The poor robustness of the air combat autonomous trajectory planning strategy (ATP) trained through vanilla reinforcement learning (RL) methods is attributed to its dependence on the source environment. When this ATP is transformed from the source to the target environment, it may fail to lead to an effective dogfight victory due to disturbances, which can potentially threaten the flight safety of the unmanned combat aerial vehicle (UCAV). In order to enhance the robustness of ATP, we propose a robust RL-based method for generating an ATP strategy. This method can effectively consider uncertainties in the state and potential model misspecifications. First, a jointly trained adversary is designed to apply disturbances in the environment, forming a two-player zero-sum game with the ATP agent. To solve this game problem and learn a robust ATP, the robust adversarial RL method (RARL) is applied. Additionally, in order to prevent non-convergence of strategies resulting from the introduction of the RARL algorithm, a curriculum learning method is proposed that emulates the way human pilots learn, gradually progressing from simpler to more challenging courses. Finally, a quantitative method for evaluating the robustness of the ATP is proposed, taking into consideration the differences in ATP performance between the source and target combat environments. The results of robustness estimations demonstrate that the RARL algorithm, which is proposed herein, effectively enhances the ATP's robustness. [ABSTRACT FROM AUTHOR]

The infinite-horizon zero-sum game of a linear system can be resorted to solve a Game algebraic Riccati equation (GARE) with indefinite quadratic term. Double-loop policy iteration algorithm is often used to find the solution of such GARE, but its calculation is usually time-consuming. In this work, we propose a novel model-based single-loop policy iteration algorithm to solve GARE and the convergence of the algorithm is guaranteed by the boundness of the iterative sequence and the comparison result. Furthermore, we devise a data-driven single-loop policy iteration algorithm for solving linear zero-sum games, without requiring the knowledge of system dynamics. Compared to the existing Newton's single-loop methods, the initialization of our algorithms is significantly relaxed and easier to implement. Two numerical examples are included to illustrate the proposed algorithms. [ABSTRACT FROM AUTHOR]

• A data-driven waterlogging risk mapping method was proposed based on long-tail learning and game theory. • A novel F (α , β) score was defined to both consider precision and recall and trade-off equity and efficiency in public emergency resource allocation. • A framework for evaluating and enhancing urban waterlogging risk mitigation capabilities was proposed, targeting nonprofit operations and government bailouts. Because disaster reduction resources are always constrained, risk mitigation measures and policies need to be weighted toward efficiency or equity. Greater efficiency focuses on high-risk areas, and greater equity focuses on identifying additional potential risk areas. To address this efficiency-equity tradeoff decision, a data-driven urban waterlogging risk mitigation capability evaluation and improving framework was proposed for nonprofit operations and government bailouts. First, a novel evaluation index was defined, the F (α , β) score, to consider both precision and recall in the binary classification tasks and to seek an effective efficiency-equity trade-off, after which a two-person, zero-sum matrix game model was developed to calculate the α and β parameters and select the highest valued optimal model. Then, using this model, a risk probability map was generated for Wuhan, China, and data envelopment analysis was used to conduct a risk mitigation capability evaluation and identify the needed policy improvements. It was found that: (1) vulnerability was the primary risk-related factor, followed by adaptability and restorability; (2) more importance should be attached to equity than efficiency when implementing risk reduction capability evaluations, that is, authorities should reduce resource investments in extremely high-risk DEA-inefficient areas and identify a greater number of potential risk areas; (3) there were around 1.55% of extremely high-risk areas, which were mainly concentrated in the Jianghan and Wuchang districts and around 9% of high-risk areas, mainly located in the Hongshan district; and (4) northeastern and southwestern Jianghan district could be a valuable reference for urban waterlogging management and disaster reduction. However, excessive investment in transportation infrastructure and construction in DEA inefficient areas should be appropriately reduced. [ABSTRACT FROM AUTHOR]

In this paper, the zero-sum game problem for linear discrete-time Markov jump systems is solved by two novel model-free reinforcement Q -learning algorithms, on-policy Q -learning and off-policy Q -learning. Firstly, under the framework of the zero-sum game, the game-coupled algebraic Riccati equation is derived. On this basis, subsystem transformation technology is employed to decouple the jumping modes. Then, a model-free on-policy Q -learning algorithm is introduced in the zero-sum game architecture to obtain the optimal control gain by measured system data. However, the probing noise will produce biases in on-policy algorithm. Thus, an off-policy Q -learning algorithm is proposed to eliminate the effect of probing noise. Subsequently, convergence is discussed for the proposed methods. Finally, an inverted pendulum system is employed to verify the validity of the proposed methods. • The zero-sum game problem for linear discrete-time Markov jump systems is addressed with the reinforcement learning method. • A novel model-free parallel off-policy Q-learning method is developed, which is not affected by probing noise. • The subsystem transformation technology is employed decoupling the jumping modes to solve the game-coupled algebraic Riccati equation. • The convergence of the proposed methods, on-policy and off-policy, for Markov jump systems are both discussed in this paper. [ABSTRACT FROM AUTHOR]

In an adversarial environment, a hostile player performing a task may behave like a non-hostile one in order not to reveal its identity to an opponent. To model such a scenario, we define identity concealment games: zero-sum stochastic reachability games with a zero-sum objective of identity concealment. To measure the identity concealment of the player, we introduce the notion of an average player. The average player's policy represents the expected behavior of a non-hostile player. We show that there exists an equilibrium policy pair for every identity concealment game and give the optimality equations to synthesize an equilibrium policy pair. If the player's opponent follows a non-equilibrium policy, the player can hide its identity better. For this reason, we study how the hostile player may learn the opponent's policy. Since learning via exploration policies would quickly reveal the hostile player's identity to the opponent, we consider the problem of learning a near-optimal policy for the hostile player using the game runs collected under the average player's policy. Consequently, we propose an algorithm that provably learns a near-optimal policy and give an upper bound on the number of sample runs to be collected. [ABSTRACT FROM AUTHOR]

Sea lice are copepod ectoparasites of major importance in salmonid aquaculture. Under controlled challenge testing, individual lice count has moderate heritability and has been suggested as a trait in genetic selection for parasite control, under the assumption that selection for reduced individual parasite burden provides group-level protection against the parasite. Recent studies indicate that genetic variation of lice count in Atlantic salmon is mostly explained by variation in initial infestation, rather than ability to limit parasite burden after infestation. Results from a selection experiment are presented, with two Atlantic salmon (Salmo salar) lines divergently selected for low/high parasite burden, showing substantial between-strain difference under common-garden testing. An experiment was performed where the final generation of the divergently selected salmon lines were challenge tested separately to assess potential for group-level protection against sea lice. The results showed that, despite the two groups being clearly different under common garden testing, the line difference was not significant when tested separately, i.e., no evidence for group-level protection. However, in a follow-up experiment, using a more realistic lice challenge model under water-flow, expected to be less favorable for the parasite, significant group-level differences were found, albeit smaller than under common garden testing. The results show that the potential for group-level protection is lower than suggested by within-group genetic variation in lice count, and more so for environments giving the parasites easy access to hosts. These results cast some doubt about the efficacy of selective breeding for reduced lice count as a tool for group-level parasite control in densely populated fish farm environments. • Distribution of sea lice over salmon hosts within an environment is heritable. • The goal of selective breeding is to reduce overall parasite infestation success. • Divergently selected groups of salmon were challenge-tested in separate tanks. • No significant genetic group-protection effect on infestation in conditions favorable for the parasite. • Moderate but significant genetic group-protection effect in less favorable conditions for the parasite. [ABSTRACT FROM AUTHOR]

In view of zero-sum differential games, this paper addresses the fault-tolerant tracking control (FTTC) problem for nonlinear systems with multiplicative actuator faults. By augmenting the nominal system, the trajectory tracking problem is transformed into an optimal regulation problem. A fault observer is designed to estimate the multiplicative actuator fault factor. In order to better reflect system performance, a comprehensive performance index function including the trajectory tracking error, the control input, and the multiplicative actuator fault is constructed. By considering the control input and the multiplicative actuator fault as two players in the zero-sum differential game (ZSDG), the FTTC problem is thus transformed into a ZSDG problem. Then, a fault-tolerant control based on ZSDG is designed by solving the Hamilton–Jacobi-Isaacs (HJI) equation. Since the HJI equation is difficult to solve, a critic neural network is constructed to obtain its approximation, which constitutes the Nash equilibrium of the ZSDG. The closed-loop faulty system is guaranteed to be uniformly ultimately bounded via the Lyapunov's direct method. The effectiveness of the developed FTTC method is demonstrated by two simulation examples. [ABSTRACT FROM AUTHOR]

In this paper, we consider a security-aware and learning-based actuator selection formulation for cyber–physical systems. A set of actuators is chosen to be used by the system so that a metric of controllability and actuation attack resilience is maximized, given only partial knowledge of the physics of the system. Such a metric is related to the trace of a game theoretic Gramian, which is obtained by solving a zero-sum game between a defender who acts as a minimizer and wants to regulate the system, and an attacker who acts as a maximizer with an eventual goal to disrupt the regulation. To account for the lack of full knowledge of the system dynamics, we design an estimator that can learn the trace of the proposed game theoretic Gramian online, by utilizing data obtained across the system's trajectories. Subsequently, we propose a scheduling algorithm that utilizes the estimator to select actuators for the system in an online manner. In fact, we prove that the set of actuators that optimize the game-based metric will be learnt and scheduled permanently in finite time. Finally, we show that the learning-based actuator placement procedure can be utilized, in its steady state, to detect a large class of actuation attacks in a partially model-free manner. We carry out a simulation on an aircraft model to show the efficacy of our framework. [ABSTRACT FROM AUTHOR]

In this tutorial we study a safety analog of the classical zero-sum differential game with positive definite penalties on the state and the two inputs. Consider a nonlinear system affine in two inputs, which are called "offender" and "defender." Let the inputs have the opposing objectives in relation to an infinite-time cost which, in addition to penalizing the inputs of both agents, incorporates a safety index of the system (a barrier function), with the defender aiming to maximize the system safety and the offender aiming to minimize it. If there is a pair of (offender, defender) non-Nash feedback policies of the L g h form with a safe outcome, namely, where the defender maintains safety while the offender fails to violate safety, then there exists an inverse optimal pair of policies that attain a Nash equilibrium relative to the safety minimax objective. In the tutorial we study both deterministic and stochastic offenders. The deterministic offender applies its feedback through its deterministic input value, while the stochastic offender applies its feedback through its incremental covariance. In addition to Nash policies for a minimax offender–defender formulation, we provide feedback laws for the defender, in the scenario where the offender action is unrestricted by optimality, and where the defender ensures input-to-state safety in the deterministic and stochastic senses. This tutorial is derived from our recent article on inverse optimal safety filters, by setting the nominal control to zero and declaring the disturbance to be the offender agent. Among several illustrative examples, one is particularly interesting and unconventional. We consider a safety game played on a unicycle vehicle between its two inputs: the angular velocity and the linear velocity, as the opposing players. We consider two scenarios. In the first, the angular velocity, acting as an offender, attempts to run the vehicle into an obstacle by steering, while the linear velocity, acting as a defender, drives the vehicle forward or in reverse to prevent the vehicle being run into the obstacle. In the second scenario, the linear velocity acts as an offender and angular velocity acts as a defender (in the deterministic case by varying the heading rate; in the stochastic case by varying the variance of a white noise driving the heading rate). A "wind" towards the obstacle advantages the offender in both scenarios. The input policies derived are optimal in the sense of their opposite objectives, under the best possible policy of the opponent, under meaningful costs on their actions. The linear velocity input prevails, whether acting in the role of a defender, in which case the collision with the obstacle is prevented, or in the role of an offender, in which case the collision with the obstacle is achieved. [ABSTRACT FROM AUTHOR]

Recent work in the field of game theoretic modeling of natural language quantifiers introduces randomizing game rules within the setting of Giles's game for Łukasiewicz logic [3,15,16]. The randomization comes about in the form of a non-strategic player, called Nature. This can, taken as a mere selection principle, lead to an immense increase of expressivity, compared to the usual two player zero sum games of perfect information, like Hintikka's game or Giles's game [3]. We will define several games with randomizing game rules and investigate their expressivity, ending with a game which we call the NRG -game, with corresponding logic Ł α (Π). We give representation theorems for the class of semi-fuzzy deliberate choice quantifiers, and, as the main result of the paper, show how one can define the operations of Gödel and Product logic, and furthermore, those of all fuzzy logics, that are based on a continuous t-norm that is representable as a finite ordinal sum of the t-norms corresponding to Łukasiewicz, Gödel and Product logic [4] , within Ł α (Π) , as long as the domains of the interpretations are finite. [ABSTRACT FROM AUTHOR]

• Investigate a novel TrFBWM to yield the TrF criteria weights. • Develop the TrFZSG to rank the alternatives. • Propose a TrF-BWM-ZSG by fusinging the BWM and ZSG. This paper investigates a hybrid emergency decision-making (EDM) method called TrF-BWM-ZSG by combining the BWM (best-worst method) and ZSG (zero-sum game) in trapezoidal fuzzy (TrF) environment. Firstly, this paper defines the entropy and similarity for trapezoidal fuzzy numbers (TrFNs). Based on the entropy and similarity measures, a bi-objective programming model is constructed to distribute the weights for experts. Then, based on the multiplicative consistent trapezoidal fuzzy reference comparisons (TrFRCs), a novel TrFBWM is investigated to yield the TrF criteria weights. This novel TrFBWM highlights two innovations: (i) A crisp goal programming weight-determining model is constructed by using the multiplicative consistent TrFRCs; (ii) A reasonable consistency ratio is defined, in which the consistency index is determined according to the consistent transitivity of TrFRCs. Afterwards, the TrFZSG is developed to achieve the alternatives' ranking order by extending the classical ZSG into TrF environment. Lastly, the availability of the TrF-BWM-ZSG is corroborated through a real EDM case of Henan rainstorm disaster. The advantages of the TrF-BWM-ZSG are manifested by several valuable comparisons. Some desirable results derived by the TrF-BWM-ZSG are received: (i) The optimal alternative selected by the proposed TrF-BWM-ZSG suits the actual rescue situation; (ii) The proposed TrF-BWM-ZSG owns high generality since TrFN can reduce to crisp number, interval and triangular fuzzy number; (iii) The developed novel TrFBWM is helpful to preserve more inherent information of TrFNs. [ABSTRACT FROM AUTHOR]

The lack of effective defense resource allocation strategies and reliable multi-agent collaboration mechanisms lead to the low stability of Deep Reinforcement Learning (DRL)-based security defense strategies in several Internet of Things (IoT) applications. To address the aforementioned issues and approach real-world scenarios, we construct a grid-based adversarial security scenario, propose a two-stage zero-sum security game model (including the resource allocation stage and the patrolling detection stage), and design a t wo-stage s ecurity g ame s olution algorithm based on DRL for this game model, named TSGS. In the resource allocation stage, TSGS uses auxiliary action embedding and gradient approximation approach to compute the Nash Equilibrium (NE) allocation strategy, which addresses the problem of unreasonable allocation. In the patrolling detection stage, TSGS achieves team collaboration by training a multi-agent Dueling Deep-Q Network under the centralized training and decentralized execution (CTDE) framework, which solves the cooperation problem among multiple defense agents. In addition, we design and implement a multi-parameterized attacker model to make the attacker's behaviors more realistic in the game. Finally, the validity of the TSGS is verified by detailed experimental results for several adversarial experimental scenarios. Compared with the baseline methods, the defense strategy learned by TSGS has higher utility and greater robustness. Especially, TSGS achieves efficient collaboration during the patrolling detection stage after resource allocation by making full use of real-time information and communication. [ABSTRACT FROM AUTHOR]

We study the behavior of a stochastic variant of replicator dynamics in two-agent zero-sum games. Inspired by the well studied deterministic replicator equation, this model gives arguably a more realistic description of real world settings and, as we demonstrate here, exhibits totally distinct behavior when compared to its deterministic counterpart which is known to be recurrent. In more detail, we characterize the statistics of such systems by their invariant measures which can be shown to be entirely supported on the boundary of the space of mixed strategies. Depending on the noise strength we can furthermore characterize these invariant measures by finding accumulation of mass at specific parts of the boundary. In particular, regardless of the magnitude of noise, we show that any invariant probability measure is a convex combination of Dirac measures on pure strategy profiles, which correspond to vertices/corners of the agents' simplices. Thus, in the presence of stochastic perturbations, even in the most classic zero-sum settings, such as Matching Pennies, we observe a stark disagreement between the axiomatic prediction of Nash equilibrium and the evolutionary emergent behavior derived by an assumption of stochastically adaptive, learning agents. • Relevant model of stochastic replicator dynamics in two-agent zero-sum games. • Characterization of game dynamics via Nash equilibria and anti-equilibria. • Classification of noise-dependent behavior via attracting invariant measures. • Stark contrast to deterministic counterpart in terms of concentration on the boundary. • Exemplification of results for games with and without interior equilibria. [ABSTRACT FROM AUTHOR]

In this work, a new tracking control scheme is developed for underactuated autonomous underwater vehicle(UAUV) with unknown disturbance. First, an error tracking system is constructed using backstepping method, which transforms the tracking control problem into the two-player zero-sum game problem. One player is the optimal controller which is designed to make the UAUV track the desired trajectory. The other player is the unknown disturbance that represent the overall damping effect of the UAUV. Then, the single critic network based online policy iteration algorithm is designed to get the optimal control law and the worst-case disturbance, which can achieve the near-optimal control performance and relax the requirements for initial admissible control conditions. In order to improve the converge velocity of tracking error, the weight update law are designed. Furthermore, a discount coefficient is introduced into the performance index due to the nonlinearity and the complexity of UAUV. In addition, the stability of the UAUV is analyzed based on the Lyapunov stability theory the system is guaranteed to be uniformly ultimately bounded (UUB). Finally, the effectiveness of the proposed method is demonstrated through the real-time simulations on the UAUV model. • The tracking control problem is solved for UAUV with unknown disturbances based on an online zero-sum game theory proposed. • Only one critic network is employed to reduce the computation time and convergence time. • An online policy iteration algorithm and weights update law are designed to solve the online zero-sum game problem. • A discount coefficient is introduced into the performance index due to the nonlinearity and complexity of UAUV. [ABSTRACT FROM AUTHOR]

Since the multi-time scales and strong nonlinearity are objectively existed in the practical industrial processes, it is extremely challenging to design control policy for optimizing the industrial operation. Therefore, this paper focuses on the multi-time scale optimal operational control (OOC) problem with unmodeled dynamics. Under the framework of reinforcement learning (RL) technology, a self-learning model-free method by integrating zero-sum game and singular perturbation (SP) theories is proposed in this paper, such that composite controllers with signal compensation can be found under which the suboptimal operation of the whole process can be achieved using only measured data. The remarkable highlight in this paper is the novel design method of the composite controllers with signal compensations for unknown nonlinear systems with multi-rate operations. Finally, the effectiveness of the proposed method is verified by a practical example and a numerical example. [ABSTRACT FROM AUTHOR]

• The fault-tolerant tracking control problem is converted into a zero-sum game problem. • The Bellman equation and the Riccati equation can be established. • An off-policy Q-learning algorithm is obtained. Traditional model-based control methods are often not applicable in industrial processes given the typical situation that model parameters are unknown, coefficient matrices are difficult to obtain, and system states are unpredictable. Accordingly, an output feedback fault-tolerant control method based on zero-sum game theory and off-policy Q-learning is presented in this study, with the aim of achieving smooth operation and good tracking performance for industrial processes that often contain sensor faults and disturbances. The specific steps are as follows. First, a system tracking error is introduced into the system to realize a novel extended model. Second, by establishing a performance index function and combining it with minimax theory, the fault-tolerant tracking control problem is converted into a zero-sum game problem. The Bellman and Riccati equations can be established after analyzing the relationship between the performance index and value functions. Then, the Q-function is introduced, and an off-policy Q-learning algorithm is combined with the Kronecker product without knowledge of system model parameters to design an optimal controller unbiased to detection noise. Finally, the effectiveness of the algorithm is verified by considering the injection molding process as an example. The experimental results validate that the designed controller demonstrates good control and extends the range of tolerable faults while maintaining good tracking performance. [ABSTRACT FROM AUTHOR]

Conflicts among various spatial planning policies have created a spatial game over land function and utilization due to the limitation of land resources. We quantitatively assess the evolutionary game process among three functional space types (agricultural production, urban–rural living, and blue–green ecological) using the concept of game theory during the period of 1980–2020 in Henan Province, China, as a case study. A research framework for assessing the spatial game process was established, and a measurement method based on spatial game type (SGT) and spatial game intensity (SGI) is proposed. The results demonstrate that the evolutionary game process among three functional space types exhibited significant spatiotemporal variation and spatial agglomeration. SGI increased from low to high values and then decreased again, and exhibited spatial autocorrelation. SGT supported classification into positive-sum and zero-sum games. The period of 1990–2000 had the largest SGI and the most counties with high-intensity zero-sum games. A highly urbanized urban center showed a high-intensity zero-sum game at the expense of agricultural production spaces. Balancing the areas devoted to various uses, enhancing spatial connectivity and improving functional mixing degree of land use may be effective ways to alleviate noncooperative games and spatial conflicts among these three functional space types. • A research framework and algorithm for the quantification of the evolutionary game process among three functional space types were proposed. • Spatial game intensity (SGI) and spatial game type (SGT) were introduced to reflect the dynamics of spatial game process. • Both SGI and SGT exhibited distinct spatiotemporal heterogeneity and spatial agglomeration. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel deep learning approach for predicting saddle points in stochastic two-player zero-sum games. Our method combines neurodynamic optimization and deep neural networks. First, we model the stochastic two-player zero-sum game as an ordinary differential equation (ODE) system using neurodynamic optimization. Second, we develop a neural network to approximate the solution to the ODE system, which includes the saddle point prediction for the game problem. Third, we introduce a specialized algorithm for training the neural network to enhance the accuracy of the saddle point prediction. Our experiments demonstrate that our model outperforms existing approaches, yielding faster convergence and more accurate saddle point predictions. [ABSTRACT FROM AUTHOR]

This paper investigates the fuzzy approximation-based optimal consensus control problem for nonlinear multiagent systems with unknown perturbations. By constructing local error dynamics, the considered optimal consensus problem is reformulated as finding Nash-equilibrium solutions to zero-sum games. Then, by using sliding mode control technology and the concept of hierarchical design, a series of control signals are sequentially designed to regulate the consensus error and minimize the local value function. In addition, an identifier-critic architecture is developed by using generalized fuzzy hyperbolic models, where the identifier is employed to relax the requirement for complete system dynamics information, and the hierarchical sliding mode surface-based critic network is applied to approximate optimal control inputs. Finally, A simulation example is presented to illustrate the validity of the proposed approach. [ABSTRACT FROM AUTHOR]

Abstract We present a novel approach to solve and create a two person zero sum matrix game by using matrix norms. Especially, we show how to obtain approximated game value for any zero sum matrix game without solving any equations using our approaches. We firstly, give the results of the lemmas for the game value depend on the matrix norms of the payoff matrix and some constants k containing the game value v. Then, we introduce row-wise and column-wise induced matrix for the payoff matrix. Moreover, we improve our approaches and present some new theorems for the game value to obtain some inequalities which depend on only the 1 − n o r m and ∞ − n o r m of the payoff matrix. Furthermore, we state the min–max theorem for p max and p min which are the maximum and minimum elements of the mixed strategy set, respectively. Finally, we illustrate and show the consistency of our approaches with some test examples. To the best of our knowledge, this is the first study in the literature that is used the matrix norms in game theory. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of fault tolerant control (FTC) for reentry vehicles (RV) subject to actuator fault and perturbation. In contrast to the majority of papers assuming bounded and exogenous actuator fault effect, the adaptive dynamic programming (ADP) control algorithm has been considered here to tackle with the dependence of closed-loop states induced by the setup of an active FTC strategy. By this way, the FTC problem formulation takes theoretically into account the fact that an actuator fault can destabilize a closed-loop. First, the FTC problem in attitude tracking is transformed into an error-optimal control problem. Next, a cost function based on zero-sum game theory is considered to achieve tracking under the joint influence of state-dependent actuator faults and perturbations. Lyapunov theory has been next used to prove the stability of FTC architecture and the network weight convergence. Benefits of proposed ADP-FTC algorithm is finally highlighted on a numerical benchmark of RV. [ABSTRACT FROM AUTHOR]