Search

In this paper, the potential differential game concept introduced by Fonseca-Morales and Hernández-Lerma (2018) is used in analyzing stabilization problems for n-player noncooperative capital accumulation games (CAGs). By first identifying a CAG as a potential game, an associated optimal control problem (OCP) of the CAG is obtained, whose optimal solution is an open-loop Nash equilibrium for the CAG. Compared with a saddle-point stability condition obtained for undiscounted CAG in the literature, a sufficient and easily verifiable condition is obtained for both discounted and undiscounted CAGs. In addition, the concept allows the turnpike property obtained for OCPs in Trélat and Zuazua (2015) to be verified for CAGs. Lastly, an illustrative example is given to verify the latter stability result for some CAGs. [ABSTRACT FROM AUTHOR]

In this paper, stabilization problems for n- player noncooperative differential games of international pollution control (IPC) are analysed via the concept of the potential differential game (PDG) introduced by Fonseca-Morales and Hernández-Lerma (2018). By first identifying a game of IPC as a PDG, an associated optimal control problem (OCP) is obtained, whose optimal solution is a Nash equilibrium (NE) for the game of IPC. Thus, the problem of finding conditions for which the NE stabilizes the game of IPC reduces to finding conditions for which the optimal solution stabilizes the associated OCP. The concept not only yields mild conditions for saddle point stability analysed in the literature but also for the overtaking optimality of the NE of the game of IPC. [ABSTRACT FROM AUTHOR]

This paper concerns noncooperative difference games with infinite horizon discounted payoffs. More precisely, it is a sequel to a previous paper [A. Fonseca-Morales and O. Hern'andez-Lerma, Potential differential games, Dynamic Games Appl., 8(2) (2018), pp. 254–279.] where the notion of continuous-time potential games was introduced. That is, a noncooperative differential game to which we can associate a continuous-time optimal control problem (OCP) whose solutions are Nash equilibria for the original game. Thus, finding or analysing the properties of Nash equilibria for the game reduces to that of the optimal solution of an OCP. Here, we study difference games, that is, the discrete-time case. First, we give several mild conditions for which a difference game is a potential difference game (PDG). Then, we illustrate our results with several examples and applications. [ABSTRACT FROM AUTHOR]

This paper introduces the notion of a potential differential game (PDG), which roughly put is a noncooperative differential game to which we can associate an optimal control problem (OCP) whose solutions are Nash equilibria for the original game. If this is the case, there are two immediate consequences. Firstly, finding Nash equilibria for the game is greatly simplified, because it is a lot easier to deal with an OCP than with the original game itself. Secondly, the Nash equilibria obtained from the associated OCP are automatically “pure” (or deterministic) rather than “mixed” (or randomized). We restrict ourselves to open-loop differential games. We propose two different approaches to identify a PDG and to construct a corresponding OCP. As an application, we consider a PDG with a certain turnpike property that is obtained from results for the associated OCP. We illustrate our results with a variety of examples. [ABSTRACT FROM AUTHOR]

NULL [ABSTRACT FROM AUTHOR]

In this paper we study the stability of the replicator dynamics for symmetric games when the strategy set is a separable metric space. In this case the replicator dynamics evolves in a space of measures. We study stability criteria with respect to different topologies and metrics on the space of probability measures. This allows us to establish relations among Nash equilibria (of a certain normal form game) and the stability of the replicator dynamics in different metrics. Some examples illustrate our results. [ABSTRACT FROM AUTHOR]

Pareto optimality and Nash equilibrium are two standard solution concepts for cooperative and non-cooperative games, respectively. At the out-set, these concepts are incompatible-see, for instance, [7] or [10]. But, on the other hand, there are particular games in which Nash equilibria turn out to be Pareto{optimal [1], [4], [6], [18], [20]. A class of these games has been identified in the context of discrete-time potential games [13]. In this paper we introduce several classes of deterministic and stochastic potential differential games [12] in which open-loop Nash equilibria are also Pareto optimal. [ABSTRACT FROM AUTHOR]

Potential games are noncooperative games for which there exist auxiliary functions, called potentials, such that the maximizers of the potential are also Nash equilibria of the corresponding game. Some properties of Nash equilibria, such as existence or stability, can be derived from the potential, whenever it exists. We survey different classes of potential games in the static and dynamic cases, with a finite number of players, as well as in population games where a continuum of players is allowed. Likewise, theoretical concepts and applications are discussed by means of illustrative examples. [ABSTRACT FROM AUTHOR]

We make a review of several variants of ergodicity for continuous-time Markov chains on a countable state space. These include strong ergodicity, ergodicity in weighted-norm spaces, exponential and subexponential ergodicity. We also study uniform exponential ergodicity for continuous-time controlled Markov chains, as a tool to deal with average reward and related optimality criteria. A discussion on the corresponding ergodicity properties is made, and an application to a controlled population system is shown. [ABSTRACT FROM AUTHOR]

In this paper, we present some new properties of the Mitra-Wan forestry model written as a discrete-time optimal control problem. For this problem, the set of stationary states is characterized. For the optimal long-run management, we consider the following optimality criteria: average optimality, good control policies, bias optimality, and overtaking optimality. We establish relationships between these criteria and show that the value of average optimal policies is constant and equals the value in the optimal stationary state. [ABSTRACT FROM AUTHOR]

This article concernsn-dimensional controlled diffusion processes. The main problem is to maximize a certain long-run average reward (also known as an ergodic reward) in such a way that a given long-run average cost is bounded above by a constant. Under suitable assumptions, the existence of optimal controls for such constrained control problems is a well-known fact. In this article we go a bit further and our goal is to introduce a technique to compute optimal controls. To this end, we follow the Lagrange multipliers approach. An example on a linear-quadratic system illustrates our results. [ABSTRACT FROM AUTHOR]

This paper concerns a class of nonstationary discrete-time stochastic noncooperative games. Our goals are threefold. First, we give conditions to find Nash equilibria by means of the Euler equation approach. Second, we identify subclasses of dynamic potential games. Finally, within one of this subclasses, we identify a further subclass for which Nash equilibria are also Pareto (or cooperative) solutions. [ABSTRACT FROM AUTHOR]

In this paper, we study the optimal ergodic control problem with minimum variance for a general class of controlled Markov diffusion processes. To this end, we follow a lexicographical approach. Namely, we first identify the class of average optimal control policies, and then within this class, we search policies that minimize the limiting average variance. To do this, a key intermediate step is to show that the limiting average variance is a constant independent of the initial state. Our proof of this latter fact gives a result stronger than the central limit theorem for diffusions. An application to manufacturing systems illustrates our results. [ABSTRACT FROM PUBLISHER]

In this paper, we study an inverse optimal problem in discrete-time stochastic control. We give necessary and sufficient conditions for a solution to a system of stochastic difference equations to be the solution of a certain optimal control problem. Our results extend to the stochastic case the work of Dechert. In particular, we present a stochastic version of an important principle in welfare economics. [ABSTRACT FROM PUBLISHER]

This article is concerned with the limiting average variance for discrete-time Markov control processes in Borel spaces, subject to pathwise constraints. Under suitable hypotheses we show that within the class of deterministic stationary optimal policies for the pathwise constrained problem, there exists one with a minimal variance. [ABSTRACT FROM AUTHOR]

This article concerns controlled Markov-modulated diffusions (also known as piecewise diffusions or switching diffusions or diffusions with Markovian switchings). Our main objective is to give conditions for the existence and characterization of overtaking optimal policies. To this end, first, we use fact that the average reward Hamilton–Jacobi–Bellman equation gives that the family of the so-called canonical control policies is nonempty. Then, within this family, we search policies with some special feature, for instance, canonical policies that in addition maximize the bias, which turn out to be overtaking optimal. [ABSTRACT FROM AUTHOR]

This paper deals with zero-sum stochastic differential games with long-run average payoffs. Our main objective is to give conditions for existence and characterization of bias and overtaking optimal equilibria. To this end, first we characterize the family of optimal average payoff strategies. Then, within this family, we impose suitable conditions to determine the subfamilies of bias and overtaking equilibria. A key step to obtain these facts is to show the existence of solutions to the average payoff optimality equations. This is done by the usual 'vanishing discount' approach. Finally, a zero-sum game associated to a certain manufacturing process illustrates our results. [ABSTRACT FROM AUTHOR]

This paper concerns two-person zero-sum games for a class of average-payoff continuous-time Markov processes in Polish spaces. The underlying processes are determined by transition rates that are allowed to be unbounded, and the payoff function may have neither upper nor lower bounds. We use two optimality inequalities to replace the so-called optimality equation in the previous literature. Under more general conditions, these optimality inequalities yield the existence of the value of the game and of a pair of optimal stationary strategies. Under some additional conditions we further establish the optimality equation itself. Finally, we use several examples to illustrate our results, and also to show the difference between the conditions in this paper and those in the literature. In particular, one of these examples shows that our approach is more general than all of the existing ones because it allows nonergodic Markov processes. [ABSTRACT FROM AUTHOR]

This article is about nonstationary nonlinear discrete-time deterministic and stochastic control systems with Borel state and control spaces, possibly noncompact control constraint sets, and unbounded costs. The control problem is to minimise an infinite-horizon total cost performance index. Using dynamic programming arguments we show that, under suitable assumptions, the optimal cost functions satisfy optimality equations, which in turn give a procedure to find optimal control policies. [ABSTRACT FROM AUTHOR]

This paper deals with discrete-time Markov control processes in Borel spaces, with unbounded rewards. The criterion to be optimized is a long-run sample-path (or pathwise) average reward subject to constraints on a long-run pathwise average cost. To study this pathwise problem, we give conditions for the existence of optimal policies for the problem with “expected” constraints. Moreover, we show that the expected case can be solved by means of a parametric family of optimality equations. These results are then extended to the problem with pathwise constraints. [ABSTRACT FROM AUTHOR]

This paper deals with denumerable-state continuous-time controlled Markov chains with possibly unbounded transition and reward rates. It concerns optimality criteria that improve the usual expected average reward criterion. First, we show the existence of average reward optimal policies with minimal average variance. Then we compare the variance minimization criterion with overtaking optimality. We present an example showing that they are opposite criteria, and therefore we cannot optimize them simultaneously. This leads to a multiobjective problem for which we identify the set of Pareto optimal policies (also known as nondominated policies). [ABSTRACT FROM AUTHOR]

This article deals with expected average reward (a.k.a. ergodic reward) and sensitive discount criteria for a general class of Markov diffusion processes. We give conditions under which: (1) average reward optimality and strong - 1-discount optimality are equivalent; (2) strong 0-discount optimality implies bias optimality; and (3) bias optimality implies 0-discount optimality. Moreover, under additional hypotheses, (4) bias optimality also implies strong 0-discount optimality. Thus, under previous results that guarantee average and bias optimality, we ensure that strong - 1-discount and strong 0-discount optimality hold. [ABSTRACT FROM AUTHOR]

This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t≥0) and conservative matrix Q( t)=[ q ij ( t)] of nonhomogeneous transition rates q ij ( t) and use it to construct the transition probability matrix. Here we obtain the same result except that the q ij ( t) are only required to satisfy a mild measurability condition, and Q( t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix, and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, and this was the main motivation for this work. [ABSTRACT FROM AUTHOR]

In this paper we give conditions for (the existence and) several characterizations of overtaking optimal policies for a general class of controlled diffusion processes. Our characterization results are of a lexicographical type; namely, first we identify the class of so-called canonical policies, and then within this class we search for policies with some special feature—for instance, canonical policies that in addition maximize the bias. [ABSTRACT FROM AUTHOR]

This paper is a survey of recent results on continuous-time Markov decision processes (MDPs) with unbounded transition rates, and reward rates that may be unbounded from above and from below. These results pertain to discounted and average reward optimality criteria, which are the most commonly used criteria, and also to more selective concepts, such as bias optimality and sensitive discount criteria. For concreteness, we consider only MDPs with a countable state space, but we indicate how the results can be extended to more general MDPs or to Markov games. [ABSTRACT FROM AUTHOR]

In this paper we consider a simulated annealing algorithm for multiobjective optimization problems. With a suitable choice of the acceptance probabilities, the algorithm is shown to converge asymptotically, that is, the Markov chain that describes the algorithm converges with probability one to the Pareto optimal set. [ABSTRACT FROM AUTHOR]

This paper studies the Monge-Kantorovich mass transfer (MT) problem on metric spaces, with possibly unbounded "cost" function. Conditions are given under which the MT problem is solvable and, furthermore, an optimal solution can be obtained as the weak limit of a sequence of optimal solutions to suitably approximating MT problems. [ABSTRACT FROM AUTHOR]

In this paper we give conditions for the existence of bias optimal policies in a class of continuous-time controlled Markov chains with unbounded reward and transition rates. Several characterizations of bias optimality are proposed. We also introduce new sets of conditions ensuring uniform exponential ergodicity of continuous-time controlled Markov chains. [ABSTRACT FROM AUTHOR]

In this paper, we consider constrained noncooperative N-person stochastic games with discounted cost criteria. The state space is assumed to be countable and the action sets are compact metric spaces. We present three main results. The first concerns the sensitivity or approximation of constrained games. The second shows the existence of Nash equilibria for constrained games with a finite state space (and compact actions space), and, finally, in the third one we extend that existence result to a class of constrained games which can be “approximated” by constrained games with finitely many states and compact action spaces. Our results are illustrated with two examples on queueing systems, which clearly show some important differences between constrained and unconstrained games. [ABSTRACT FROM AUTHOR]

This paper concerns the existence and characterization of optimal randomized strategies for some constrained optimization and control problems. We first present a characterization of the extreme points of a set of randomized strategies that satisfy n moment-like constraints. Conditions are given under which those extreme points are randomizations of at most n+1 deterministic strategies. This result is then applied to obtain the existence and characterization of optimal strategies for a class of deterministic, allocation-like, optimization problems and their Young relaxations. Similar results are obtained for constrained Markov control processes in Borel spaces. [ABSTRACT FROM AUTHOR]

This paper deals with continuous-time zero-sum two-person Markov games with denumerable state space, general (Borel) action spaces and possibly unbounded transition and reward/cost rates. We analyze the bias optimality and the weakly overtaking optimality criteria. An example shows that, in contrast to control (or one-player) problems, these criteria are not equivalent for games. [ABSTRACT FROM AUTHOR]

In this paper, we study two-person nonzero-sum games for continuous-time Markov chains with discounted payoff criteria and Bore! action spaces. The transition rates are possibly unbounded, and the payoff functions might have neither upper nor lower bounds. We give conditions that ensure the existence of Nash equilibria in stationary strategies. For the zero-sum case, we prove the existence of the value of the game, and also provide a recursive way to compute it, or at least to approximate it. Our results are applied to a controlled queueing system. We also show that if the transition rates are uniformly bounded, then a continuous-time game is equivalent, in a suitable sense, to a discrete-time Markov game. [ABSTRACT FROM AUTHOR]

This paper gives conditions for the convergence of the Laurent series expansion for a class of continuous-time controlled Markov chains with possibly unbounded reward (or cost) rates and unbounded transition rates. That series is then used to study several optimization criteria, includingn-discount optimality (forn=-1,0,1,...), Blackwell optimality, and the maximization of a certain vector criterion that in particular gives gain and bias optimality. [ABSTRACT FROM AUTHOR]

This paper concerns discrete-time multiobjective Markov control processes on Borel spaces and unbounded costs. Under mild assumptions, it is shown that the usual “scalarization approach” to obtain Pareto policies for the multiobjective control problem is in fact equivalent to solving the dual of a certain multiobjective infinite-dimensional linear program. The latter program is obtained from a multiobjective measure problem which is also used to prove the existence of strong Pareto policies, that is, Pareto policies whose cost vector is the closest to the control problem’s virtual minimum. [ABSTRACT FROM AUTHOR]

In this paper we consider a general optimization problem (OP) and study the convergence and approximation of optimal values and optimal solutions to changes in the cost function and the set of feasible solutions. By a "general" OP we mean that the cost function and the constraints are defined on a Hausdorff topological space. First we obtain convergence results for a general OP, and then we present an application of these results on the approximation of the optimal value and the optimal solutions for the so-called general capacity problem in metric spaces. [ABSTRACT FROM AUTHOR]

This paper considers constrained Markov control processes in Borel spaces, with unbounded costs. The criterion to be minimized is a long-run expected average cost, and the constraints can be imposed on similar average costs, or on average rewards, or discounted costs or rewards. We give conditions under which the constrained problem (CP) is solvable and equivalent to an equality constrained (EC) linear program. Furthermore, we show that there is no duality gap between EC and the dual program EC* and that in fact the strong duality condition holds. Finally, we introduce an explicit procedure to solve CP in some cases which is illustrated with a detailed example. [ABSTRACT FROM AUTHOR]

In this paper we study constrained continuous-time Markov control processes with a denumerable state space and unbounded reward/cost and transition rates. The criterion to be maximized is an expected discounted reward, and the constraint is imposed on an expected discounted cost. We give conditions that ensure the existence of constrained-optimal policies. We also show that a constrained-optimal policy may be a stationary policy or a randomized stationary policy that randomizes between two stationary policies which differ in at most one state. Our results are illustrated with a controlled queueing system. [ABSTRACT FROM AUTHOR]

We consider a sequence of discounted cost, constrained Markov control processes (CCPs) with countable state space, metric action set and possibly unbounded cost functions. We give conditions under which the sequence of optimal values of the CCPs converges to the optimal value of a limiting CCP, and, furthermore, the accumulation points of sequences of optimal policies for the CCPs are optimal policies for the limiting CCP. These results are obtained via an approximation theorem for general minimization problems. [ABSTRACT FROM AUTHOR]

This paper studies the Monge–Kantorovich mass transfer (MT) problem on metric spaces and with an unbounded cost function. Conditions are given under which the strong duality condition holds; that is, MT and its dual MT $^*$ are both solvable and their optimal values coincide. [ABSTRACT FROM AUTHOR]

In this paper, we use the Lagrange multipliers approach to study a general infinite-dimensional inequality-constrained linear program IP. The main problem we are concerned with is to show that the strong duality condition for IP holds, so that IP and its dual IP* are both solvable and their optimal values coincide. To do this, we first express IP as a convex program with a Lagrangian function L, say. Then we show that the strong duality condition implies the existence of a saddle point for L, and that, under an additional, mild condition, the converse is also true. Moreover, the saddle point gives optimal solutions for IP and IP*. Thus, our original problem is essentially reduced to prove the existence of a saddle point for L, which is shown to be the case under suitable assumptions. We use this fact to study equality-constrained programs, and we illustrate our main results with applications to the general capacity and the mass transfer problems. [ABSTRACT FROM AUTHOR]

This paper concerns nonstationary continuous-time Markov control processes on Polish spaces, with the infinite-horizon discounted cost criterion. Necessary and sufficient conditions are given for a control policy to be optimal and asymptotically optimal. In addition, under suitable hypotheses, it is shown that the successive approximation procedure converges in the sense that the sequence of finite-horizon optimal cost functions and the corresponding optimal control policies both converge. [ABSTRACT FROM AUTHOR]

This paper introduces a multiobjective control formulation of the priority assignment (PA) problem for a discrete-time single-server queueing system with q competing classes of customers, and the discounted cost criterion. A multiobjective priority assignment (MPA) problem is presented, which is analyzed from the viewpoint of both Pareto and lexicographic optimality. The cμ-rule, which is a well known optimal control policy for the PA problem, is shown to be Pareto optimal, strong Pareto optimal with respect to a certain norm on ℝ[sup q] , and also lexicographically optimal for MPA. These results are used to study a constrained PA problem. [ABSTRACT FROM AUTHOR]

Abstract. This paper concerns the general capacity (GC) problem on metric spaces. Conditions are given under which the strong duality condition holds, that is, GC and its dual GC* are both solvable and their optimal values coincide. [ABSTRACT FROM AUTHOR]

We consider constrained discounted-cost Markov control processes in Borel spaces, with unbounded costs. Conditions are given for the constrained problem to be solvable, and also equivalent to an equality-constrained (EC) linear program. In addition, it is shown that there is no duality gap between EC and its dual program EC[sup *] , and that, under additional assumptions, also EC[sup *] is solvable, so that in fact the strong duality condition holds. Finally, a Farkas-like theorem is included, which gives necessary and sufficient conditions for the primal program EC to be consistent. [ABSTRACT FROM AUTHOR]