Search

We study the inter-temporally optimal innovation strategies of incumbent manufacturing firms that compete in an established market and can extend their product line through product innovation. Firms invest in production capacity and R&D knowledge stock, where the R&D knowledge stock and the current R&D investment determine the hazard rate of innovation. Our findings show that the firms’ optimal R&D strategies are driven by a subtle interplay between the relative positions of their R&D knowledge stocks and their current relative positions on the established market. First, we find that under symmetric investment costs the knowledge leader should spend more on R&D than the knowledge laggard only if it has a substantially smaller market share on the established market. If the knowledge leader’s market share is sufficiently large, its optimal investment in R&D is so small that its innovation rate is lower than the knowledge laggard’s. Second, optimal investment in R&D knowledge is negatively affected by the opponent’s production capacity on the established market if the competitor has not innovated yet. However, we find that this effect is reversed after the competitor has successfully introduced the new product on the submarket. Third, the manufacturing firm with higher costs of adjusting production capacity for the established product has a higher incentive to engage in product innovation and might even achieve a higher total discounted profit than its more efficient competitor.

This paper considers remote state estimation in cyber-physical systems (CPSs) with multiple sensors. Each plant is modeled by a discrete-time stochastic linear system with measurements of each sensor transmitted to the corresponding remote estimator over a shared communication network when their securities are interdependent due to network induced risks. A dynamic nonzero-sum game with asymmetric information is formulated in which each sensor subject to a resource budget constraint needs to decide whether to invest in security for sending data packets, taking the behaviors of other sensors into account. To overcome the difficulty in characterizing or computing the Nash equilibria (NE), the game with asymmetric information is transformed into another game with symmetric information such that the equilibrium of the original game can be obtained by solving the equilibrium of the new game. Under certain conditions, we devise a backward induction algorithm to obtain a subclass of NE of the original game, known as common information based Markov perfect equilibria (CIBMPE). Finally, a numerical example is provided to illustrate the results obtained.

We consider a two-player game of war of attrition under complete information. It is well-known that this class of games admits equilibria in pure, as well as mixed strategies, and much of the literature has focused on the latter. We show that if the players' payoffs whilst in "war" vary stochastically and their exit payoffs are heterogeneous, then the game admits Markov Perfect equilibria in pure strategies only. This is true irrespective of the degree of randomness and heterogeneity, thus highlighting the fragility of mixed-strategy equilibria to a natural perturbation of the canonical model. In contrast, when the players' flow payoffs are deterministic or their exit payoffs are homogeneous, the game admits equilibria in pure and mixed strategies., Comment: 36 pages

International audience; We study a stochastic game with a dynamic set of players, for modeling and analyzing their computational investment strategies in distributed computing. Players obtain a certain reward for solving a problem, while incurring a certain cost based on the invested time and computational power. We present our framework while considering a contemporary application of blockchain mining, and show that the framework is applicable to certain other distributed computing settings as well. For an in-depth analysis, we consider a particular yet natural scenario where the rate of solving the problem is proportional to the total computational power invested by the players. We show that, in Markov perfect equilibrium, players with cost parameters exceeding a certain threshold, do not invest; while those with cost parameters less than this threshold, invest maximal power. We arrive at an interesting conclusion that the players need not have information about the system state as well as each others' parameters, namely, cost parameters and arrival/departure rates. With extensive simulations and insights through mean field approximation, we study the effects of players' arrival/departure rates and the system parameters on the players' utilities.

We prove for a large class of stochastic games with discontinuous payoffs, a stationary Markov perfect equilibrium exists under the condition of “continuation payoff security.” This condition is easy to verify and holds in many economic games. Roughly, a game belongs to this class if for any action/state profiles and continuation payoff, a player can identify another action at the current stage with the payoff not much worse than her current one, even if other players perturb actions slightly. As an illustrative application of the equilibrium existence result, we provide a stochastic dynamic oligopoly model of firm entry, exit, and price competitions.

Markov decision processes (MDPs) provide a powerful framework for analyzing dynamic decision making. However, their applications are significantly hindered by the difficulty of obtaining solutions. In this study, we introduce reducible MDPs whose exact solutions can be obtained by solving simpler MDPs, termed the coordinate MDPs. The value function and an optimal policy of a reducible MDP are linear functions of those of the associated coordinate MDP. Because the coordinate MDP does not involve the multi‐dimensional endogenous state, we achieve dimension reduction on a reducible MDP. Extending the MDP framework to multiple players, we introduce reducible stochastic games. We show that these games reduce to simpler coordinate games that do not involve the multi‐dimensional endogenous state. We specify sufficient conditions for the existence of a pure‐strategy Markov perfect equilibrium in reducible stochastic games and derive closed‐form expressions for the players’ equilibrium values. The reducible framework encompasses a variety of linear and nonlinear models and offers substantial simplification in analysis and computation. We provide guidelines and illustrative examples on formulating problems as reducible models. We demonstrate the applicability and modeling flexibility of reducible models in a wide range of contexts including capacity and inventory management and duopoly competition.

This paper studies optimal discretionary monetary and fiscal policy when the lower bound on nominal interest rates is occasionally binding in a model with nominal rigidities and long-term government debt. At the lower bound it is optimal for the government to temporarily reduce debt.

Managers, let stockpiling be but at a higher price—don’t hope to cut stockpiling by lopping off promotions.

In many domains, committees bargain over a sequence of policies and a policy remains in effect until a new agreement is reached. In this paper, I argue that, in order to assess the consequences of veto power, it is important to take into account this dynamic aspect. I analyze an infinitely repeated divide-the-dollar game with an endogenous status quo policy. I show that full appropriation by the veto player is the only stable policy when legislators are sufficiently impatient; and that, irrespective of legislators' patience and the initial division of resources, there is always an equilibrium where policy eventually gets arbitrarily close to full appropriation by the veto player. In this equilibrium, increasing legislators' patience or decreasing the veto player's proposal power makes convergence to this outcome slower and the veto player supports reforms that decrease his allocation. The main predictions of the theory find support in controlled laboratory experiments.

We analyze the strategic implications of consumers’ reference-price effects, either symmetric (for loss-neutral consumers) or asymmetric (for loss-averse consumers), in a differentiated oligopoly model where firms compete either in prices (a la Bertrand) or in quantities (a la Cournot) over an infinite time horizon. The dynamic game is specified in continuous time. The solution concept is Markov Perfect Equilibrium. We show how price dynamics in the presence of reference-price effects crucially depends on the nature of market competition. One of the main results of our analysis is that, with loss-averse consumers, there exists an interval of initial reference prices such that firms adopt the same constant-pricing strategy in both the Bertrand and the Cournot games, implying that the distinction between price and quantity competition has no impact on market conduct and performance.

We study Markov decision processes with Borel state spaces under quasi-hyperbolic discounting. This type of discounting nicely models human behaviour, which is time-inconsistent in the long run. The decision maker has preferences changing in time. Therefore, the standard approach based on the Bellman optimality principle fails. Within a dynamic game-theoretic framework, we prove the existence of randomised stationary Markov perfect equilibria for a large class of Markov decision processes with transitions having a density function. We also show that randomisation can be restricted to two actions in every state of the process. Moreover, we prove that under some conditions, this equilibrium can be replaced by a deterministic one. For models with countable state spaces, we establish the existence of deterministic Markov perfect equilibria. Many examples are given to illustrate our results, including a portfolio selection model with quasi-hyperbolic discounting.

We study Markov perfect equilibria (MPE) of two-player multi-mode differential games with controlled state dynamics, where one player controls the transition between modes. Different types of MPE are characterized distinguishing between delay equilibria, inducing for some initial conditions mode switches after a positive finite delay, and now or never equilibria, under which, depending on the initial condition, a mode switch occurs immediately or never. These results are applied to analyze the MPE of a game capturing the dynamic interaction between two incumbent firms among which one has to decide when to extend its product range by introducing a new product. The market appeal of the new product can be influenced over time by both firms through costly investments. Under a wide range of market introduction costs a now or never equilibrium coexists with a continuum of delay equilibria, each of them inducing a different time of product introduction.

We establish a fixed point theorem for the composition of nonconvex, measurable selection valued correspondences with Banach space valued selections. We show that if the underlying probability space of states is nonatomic and if the selection correspondences in the composition are K-correspondences (meaning correspondences having graphs that contain their Komlos limits), then the induced measurable selection valued composition correspondence takes contractible values and therefore has fixed points. As an application we use our fixed point result to show that all nonatomic uncountable-compact discounted stochastic games have stationary Markov perfect equilibria – thus resolving a long-standing open question in game theory.

We study the Markov perfect equilibrium in a dynamic game where agents have non-constant time preference, decentralized households determine aggregate savings, and a planner chooses climate policy. The article is the first to solve this problem with general discounting and general functional forms. With time-inconsistent preferences, a commitment device that allows a planner to choose climate policy for multiple periods is potentially very valuable. Nevertheless, our quantitative results show that while a permanent commitment device would be very valuable, the ability to commit policy for “only” 100 years adds less than 2% to the value of climate policy without commitment. We solve a log-linear version of the model analytically, generating a formula for the optimal carbon tax that includes the formula in Golosov et al. (2014, Econometrica, 82, 41–88) as a special case. More importantly, we develop new algorithms to solve the general game numerically. Convex damages lead to strategic interactions across generations of planners that lower the optimal carbon tax by 45% relative to the scenario without strategic interactions.

We examine the planner’s dynamic regulation problem in an emission trading system (ETS) with allowance banking. The planner sets the emissions cap for the next period after the current period allowance market has cleared, but before knowing the next period’s abatement cost realization. This creates a time consistency problem when banking is possible. We examine two policies to overcome the consistency problem: a commitment solution and the Markov perfect solution. We show that the endogenous price floor generated by the banking demand becomes an integral feature of the two policies. Hence, they can be best described as hybrid policies that combine elements from emissions taxes and tradable allowances. This reveals new welfare implications that have an influence on instrument choice in the traditional prices versus quantities setup. We compare the expected welfare outcomes of four different policy instruments: the commitment policy, the Markov policy, a Pigouvian tax, and a no-banking ETS. We show that allowing banking can yield welfare gains compared to tax and quantity regulation, with or without commitment.

Along with the diversification of electricity market, the voltage regulation (VR) service is opened up to various qualified providers to meet the enormous demand, among which, the electric vehicle (EV) aggregators (AGGs) can integrate the scattered EVs and play the role of VR sources with a high response speed and low cost. In order to coordinate the VR service providers to satisfy the total VR requirement, auction mechanisms are widely used to select VR sources to achieve better performance. Yet, it is challenging to optimize the strategy of each EV AGG in the auction process due to the stochastic EV mobility, various distribution network topology, and the competition mechanism. To address these challenges, we proposed a discounted stochastic multiplayer game (DSMG) approach to analyze the competition among EV AGGs. Due to the constraint of distribution network topology, the efficiency of the VR sources at different locations can be different. Thus, the impact of distribution network topology on the VR efficiency is investigated by DSO when evaluating the capacity of AGGs. The randomness of EV numbers is considered when predicting the AGGs’ available VR capacity so that the tendency for the AGGs to follow the optimal strategies can be modeled accurately. Accordingly, a linear power flow analysis approach and a battery pool model are developed to address the distribution network topology and EV mobility, respectively. Then, the DSMG approach is used in the VR auction process to optimize the AGGs’ strategies. The existence proof of the stationary Markov perfect equilibrium is presented, and the corresponding algorithms to obtain the equilibrium is proposed. The performance of the proposed DSMG approach is evaluated and compared with other approaches based on the IEEE 33-bus test feeder, IEEE 123-bus test feeder, and the real-world generation and load data from PVWatts Calculator and Market Analysis and Information System, respectively.

Recent work on optimal monetary and fiscal policy in New Keynesian models has tended to focus on policy set by an infinitely lived benevolent policy maker, often with access to a commitment technology. In this paper, we explore deviations from this ideal, by allowing (time-consistent) policy to be set by a process of bargaining between two political players with different weights on elements of the social welfare function. We characterize the (linear) Markov perfect equilibrium and, in a series of numerical examples, we explore the resultant policy response to shocks which cannot be perfectly offset with the available instruments due to their fiscal consequences. We find that, even although the players, on average, have the socially desirable objective function, the process of bargaining implies an outcome which deviates from the time-consistent policy chosen by the benevolent policy maker. Moreover, the range of instruments available mean that policy makers will bargain across the entire policy mix, sometimes implying outcomes which are quite different from those that would be chosen by a single policy maker. These policy outcomes depend crucially on the nature of the conflict and also the level of government debt.

Increasingly, retailers have access to better pricing technology, especially in online markets. Using hourly data from five major online retailers, we show that retailers set prices at regular intervals that differ across firms. In addition, faster firms appear to use automated pricing rules that are functions of rivals' prices. These features are inconsistent with the standard assumptions about pricing technology used in the empirical literature. Motivated by these facts, we consider a model of competition in which firms can differ in pricing frequency and choose pricing algorithms rather than prices. We demonstrate that, relative to the standard simultaneous price-setting model, pricing technology with these features can increase prices in Markov perfect equilibrium. A simple counterfactual simulation implies that pricing algorithms lead to meaningful increases in markups in our empirical setting, especially for firms with the fastest pricing technology.

We consider a two-person stochastic game of resource extraction. It is assumed that players have identical preferences. A novelty relies on the fact that each player is equipped with the same risk coefficient and calculates his discounted utility in the infinite time horizon in a recursive way by applying the entropic risk measure parametrized by this risk coefficient. Under two alternative sets of assumptions, we prove the existence of a symmetric stationary Markov perfect equilibrium.

This paper studies the problem of non-cooperative radio resource scheduling in a vehicle-to-vehicle communication network. The technical challenges lie in high vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued packets. The frequency allocation at the beginning of each slot by the RSU is regulated following a sealed second-price auction. Each VUE-pair aims to optimize the expected long-term performance. Such interactions among VUE-pairs are modelled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the stochastic game into an equivalent game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium (MPE), which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE policies. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

This paper characterizes the subgame-perfect pure-strategy equilibrium paths in discounted supergames with perfect monitoring. It is shown that all the equilibrium paths are composed of fragments called elementary subpaths. This characterization result is complemented with an algorithm for finding the elementary subpaths. By using these subpaths it is possible to generate equilibrium paths and payoffs. When there are finitely many elementary subpaths, all the equilibrium paths can be represented by a directed graph. These graphs can be used in analyzing the complexity of equilibrium outcomes. In particular, it is shown that the size and the density of the equilibrium set can be measured by the asymptotic growth rate of equilibrium paths and the Hausdorff dimension of the payoff set.

In this paper, baseball is formulated as a finite Markov game with approximately 6.45 million states. We give an effective dynamic programming algorithm which computes Markov perfect equilibria and the value functions of the game for both teams in 2 second per game. Optimal decision making can be found depending on the situation—for example, for the batting team, whether batting for a hit, stealing a base or sacrifice bunting will maximize their win percentage, or for the fielding team, whether to pitch to or intentionally walk a batter, yields optimal results. In addition, our algorithm makes it possible to compute the optimal batting order, in consideration of strategy optimization such as a sacrifice bunt or a stolen base. The authors believe that this baseball model is also useful as a benchmark instance for evaluating the performances of (multi-agent) Reinforcement Learning methods.

We examine a two-player game with two-armed exponential bandits a la (Keller et al. in Econometrica 73:39–68, 2005), where players operate different technologies for exploring the risky option. We characterise the set of Markov perfect equilibria and show that there always exists an equilibrium in which the player with the inferior technology uses a cut-off strategy. All Markov perfect equilibria imply the same amount of experimentation but differ with respect to the expected speed of the resolution of uncertainty. If and only if the degree of asymmetry between the players is high enough, there exists a Markov perfect equilibrium in which both players use cut-off strategies. Whenever this equilibrium exists, it welfare dominates all other equilibria. This contrasts with the case of symmetric players, where there never exists a Markov perfect equilibrium in cut-off strategies.

In this paper, we consider a stochastic economic growth model in the form of an intergenerational dynamic game. Both paternalistic and non-paternalistic components are present in the model. Under very general assumptions allowing for unbounded utility functions and weakly continuous transitions, we establish the existence of Markov perfect equilibria that consist of a consumption strategy and an indirect utility function. In the pure paternalistic case, we obtain new results on equilibria. An important point of our contribution is that we make no separability assumptions on the utility functions of generations.

Fixed points of the (most) refined best reply correspondence, introduced in Balkenborg et al. (2013), in the agent normal form of extensive form games with perfect recall have a remarkable property. They induce fixed points of the same correspondence in the agent normal form of every subgame. Furthermore, in a well-defined sense, fixed points of this correspondence refine even trembling hand perfect equilibria, while, on the other hand, reasonable equilibria that are not weak perfect Bayesian equilibria are fixed points of this correspondence.

A seller has a fixed number of goods to sell by a deadline. At each time, he posts a regular price and decides whether to hold a flash sale. Over time, buyers privately enter the market and strategically time their purchases. If a buyer does not purchase when she arrives, she can pay an attention cost to recheck the regular price afterwards, or she can wait for future flash sales where she may obtain a good at a discounted price. In the unique Markov perfect equilibrium, the seller sporadically holds flash sales to lower the stock of goods. A flash sale increases the willingness to pay of future buyers, but decreases the willingness to pay of buyers who arrive early in the game. When it is very likely that a buyer will obtain a good in a flash sale, the seller holds a “big” initial flash sale for all but one unit of the good.

This paper develops a simple dynamic oligopoly model and a quick estimation procedure for empirical applications.

This study presents a two-period overlapping-generations model with endogenous growth. In each period, the government representing young and old generations provides a public good financed by labor income taxation and public debt issuance, and the government's policies are determined by probabilistic voting. Increased political power of the old lowers economic growth. A debt-ceiling rule is considered to resolve the negative growth effect, but it creates a trade-off between generations in terms of welfare.

We study a tug-of-war game between two players using the lottery contest success function (CSF) and a quadratic cost (of effort) function. We construct a pure strategy symmetric Markov perfect equilibrium of this game, show that it is unique, and provide closed-form solutions for equilibrium strategies and values. In stark contrast to a model of tug-of-war with an all-pay auction CSF, players exert positive efforts until the very last battle in this equilibrium. We deliver a set of empirically appealing results on effort dynamics.

We experimentally implement a dynamic public‐good problem, where the public good in question is the dynamically evolving information about agents' common state of the world. Subjects' behavior is consistent with free‐riding because of strategic concerns. We also find that subjects adopt more complex behaviors than predicted by the welfare‐optimal equilibrium, such as noncut‐off behavior, lonely pioneers, and frequent switches of action. Dynamic public‐good problem strategic experimentation exponential bandits learning dynamic games laboratory experiments C73 C92 D83 O32

Many negotiations (for instance, among political parties or partners in a business) are characterized by dynamic bargaining: current agreements affect future bargaining possibilities. We study such situations using bargaining games á la Rubinstein (Econometrica 50:97–109, 1982), with the novelty that players can decide how much to invest, as well as how to share the residual surplus for their own consumption. Their investment decisions affect the size of the next surplus. In line with the existing literature, we focus on Markov Perfect Equilibria, where consumption and investment are linear time-invariant functions of capital and show that standard results in bargaining theory can be overturned. For instance, a more patient proposer may consume less than his opponent. The intuition is that when capital is productive, both parties have incentives to invest, however, the most patient party wishes to invest significantly more than his opponent. Then, to prioritize investment—which affects future bargaining possibilities—the former must make larger concessions and let the latter consume more. Another interesting result is that if a player becomes more patient, both parties may reduce their investment. The key underlying driver of this result is that when counteroffers become cheaper for an impatient party, he is able to reduce his investment and consume more. This forces his opponent to make larger concessions (and reduce his investment plan). Moreover, extreme demands (where a player consumes all the residual surplus) are possible in equilibrium, under fairly modest assumptions. Finally, only when bargaining is frictionless, is the equilibrium efficient.