Search

Problem definition: We consider two business models for a two-sided economy under uncertainty: dealership and marketplace with fulfillment services. Although both business models can bridge the gap between demand and supply, it is not clear which model is better for the firm or for the consumers. Methodology/results: We show that, whereas the two models differ substantially in pricing power, inventory risk, fee structure, and fulfillment time, both models share several important features with the revenues earned by the firm from the two models converging when the markets are thick. We also show that, for thick markets, there is a one-to-one mapping between their corresponding optimal policies. Managerial implications: Our results provide guidelines for firms entering two-sided markets: when the market is thick, the two business models are similar; when the market is thin, they should carefully inspect a number of market conditions before making the choice. Funding: The research of G. Li is partially supported by the National Natural Science Foundation of China [Grants 72150002, 72394361] and the Guangdong Provincial Key Laboratory of Mathematical Foundations for Artificial Intelligence [Grant 2023B1212010001]. The research of N. Chen is partially supported by the Natural Sciences and Engineering Research Council of Canada Discovery [Grant RGPIN-2020-04038]. The research of G. Gallego is partially supported by Collaborative Research Funding Hong Kong [Grant C6032-21G]. The research of P. Gao is supported by the National Natural Science Foundation of China [Grants 72201234, 72192805], Collaborative Research Funding Hong Kong [Grant C6032-21G], and the Guangdong Provincial Key Laboratory of Mathematical Foundations for Artificial Intelligence [Grant 2023B1212010001]. Supplemental Material: The online supplement is available at https://doi.org/10.1287/msom.2022.0253. [ABSTRACT FROM AUTHOR]

In this paper, we study discounted stochastic games with an additive rewards and transitions (ARAT) structure. We assume that the transition probabilities are dominated by some probability measures on a countably generated measurable state space and are norm continuous in actions of the players. We prove the existence of Markov perfect equilibria in which the players can use randomization on two pure actions in each state. When the state space is countable, then we show that there exists a pure Markov perfect equilibrium. Assuming additionally that the transition probabilities have no conditional atoms, we establish some existence results on pure Markov perfect equilibria for ARAT games with uncountable state space using a version of Lyapunov's theorem for conditional expectation of correspondences due to Dynkin and Evstigneev. We also include a wide-ranging discussion on equilibria for discounted stochastic games with a general state space. [ABSTRACT FROM AUTHOR]

We consider a distributed computing setting wherein a central entity seeks power from computational providers by offering a certain reward in return. The computational providers are classified into long-term stakeholders that invest a constant amount of power over time and players that can strategize on their computational investment. In this paper, we model and analyze a stochastic game in such a distributed computing setting, wherein players arrive and depart over time. While our model is formulated with a focus on volunteer computing, it equally applies to certain other distributed computing applications such as mining in blockchain. We prove that, in Markov perfect equilibrium, only players with cost parameters in a relatively low range which collectively satisfy a certain constraint in a given state, invest. We infer that players need not have knowledge about the system state and other players' parameters, if the total power that is being received by the central entity is communicated to the players as part of the system's protocol. If players are homogeneous and the system consists of a reasonably large number of players, we observe that the total power received by the central entity is proportional to the offered reward and does not vary significantly despite the players' arrivals and departures, thus resulting in a robust and reliable system. We then study by way of simulations and mean field approximation, how the players' utilities are influenced by their arrival and departure rates as well as the system parameters such as the reward's amount and dispensing rate. We observe that the players' expected utilities are maximized when their arrival and departure rates are such that the average number of players present in the system is typically between 1 and 2, since this leads to the system being in the condition of least competition with high probability. Further, their expected utilities increase almost linearly with the offered reward and converge to a constant value with respect to its dispensing rate. We conclude by studying a Stackelberg game, where the central entity decides the amount of reward to offer, and the computational providers decide how much power to invest based on the offered reward. [ABSTRACT FROM AUTHOR]

Using a dynamic preferential trade agreement (PTA) formation model, we show that political biases in exporting and import‐competing sectors substantially impact the extent of PTA formation. While both exclusion and free riding incentives constrain the expansion to global free trade in a free trade agreement (FTA) game, only the former incentive arises in a customs union (CU) game. When we endogenize the choice between FTAs and CUs, the tension between the flexibility benefit of FTAs and the coordination benefit of CUs reveals that FTAs always serve as PTA building blocks while CUs may serve as PTA building or stumbling blocks. [ABSTRACT FROM AUTHOR]

Similar to the role of Markov decision processes in reinforcement learning, Markov games (also called stochastic games) lay down the foundation for the study of multi-agent reinforcement learning and sequential agent interactions. We introduce approximate Markov perfect equilibrium as a solution to the computational problem of finite-state stochastic games repeated in the infinite horizon and prove its PPAD -completeness. This solution concept preserves the Markov perfect property and opens up the possibility for the success of multi-agent reinforcement learning algorithms on static two-player games to be extended to multi-agent dynamic games, expanding the reign of the PPAD -complete class. [ABSTRACT FROM AUTHOR]

We study a stochastic game with a dynamic set of players, for modeling and analyzing their computational investment strategies in distributed computing. Players obtain a certain reward for solving a problem, while incurring a certain cost based on the invested time and computational power. We present our framework while considering a contemporary application of blockchain mining, and show that the framework is applicable to certain other distributed computing settings as well. For an in-depth analysis, we consider a particular yet natural scenario where the rate of solving the problem is proportional to the total computational power invested by the players. We show that, in Markov perfect equilibrium, players with cost parameters exceeding a certain threshold, do not invest; while those with cost parameters less than this threshold, invest maximal power. We arrive at an interesting conclusion that the players need not have information about the system state as well as each others' parameters, namely, cost parameters and arrival/departure rates. With extensive simulations and insights through mean field approximation, we study the effects of players' arrival/departure rates and the system parameters on the players' utilities. • Stochastic game framework for distributed computing with application to block mining. • Players can arrive and depart over time and strategize on their power investments. • Markov perfect equilibrium follows a threshold policy. • Sensitivity analysis and insights into results using a mean field approach. [ABSTRACT FROM AUTHOR]

Copyright of Modern Economic Science is the property of Modern Economic Science Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We study Markov perfect equilibria (MPE) of two-player multi-mode differential games with controlled state dynamics, where one player controls the transition between modes. Different types of MPE are characterized distinguishing between delay equilibria, inducing for some initial conditions mode switches after a positive finite delay, and now or never equilibria, under which, depending on the initial condition, a mode switch occurs immediately or never. These results are applied to analyze the MPE of a game capturing the dynamic interaction between two incumbent firms among which one has to decide when to extend its product range by introducing a new product. The market appeal of the new product can be influenced over time by both firms through costly investments. Under a wide range of market introduction costs a now or never equilibrium coexists with a continuum of delay equilibria, each of them inducing a different time of product introduction. [ABSTRACT FROM AUTHOR]

This paper investigates how the presence of social capital affects the externality arising from status‐seeking preference as a parable for inefficient antagonistic behavior. It is assumed that the stock of social capital accumulates through the strategic interaction among a finite number of rational, infinitely‐lived individuals. Using a differential game, we show that there are two types of Markov perfect equilibrium strategy, of which one leads a society to zero social capital, while the other leads to the satiation level of social capital. When there is an unstable interior steady state, there is a threshold: with any initial stock of social capital above (below) that, society is able to build social capital (get stuck in a poverty trap of null social capital). In the latter case, the intervention of governments is called upon, because social welfare in the poverty trap is less than that in the social capital‐rich society. [ABSTRACT FROM AUTHOR]

The game of resource extraction/capital accumulation is a stochastic infinitehorizon game, which models a joint utilization of a productive asset over time. The paper complements the available results on pure Markov perfect equilibrium existence in the non-symmetric game setting with an arbitrary number of agents. Moreover, we allow that the players have unbounded utilities and relax the assumption that the stochastic kernels of the transition probability must depend only on the amount of resource before consumption. This class of the game has not been examined beforehand. However, we could prove the Markov perfect equilibrium existence only in the specific case of interest. Namely, when the players have constant relative risk aversion (CRRA) power utilities and the transition law follows a geometric random walk in relation to the joint investment. The setup with the chosen characteristics is moti-vated by economic considerations, which makes it relevant to a certain range of real-word problems. [ABSTRACT FROM AUTHOR]

Recent work on optimal monetary and fiscal policy in New Keynesian models has tended to focus on policy set by an infinitely lived benevolent policy maker, often with access to a commitment technology. In this paper, we explore deviations from this ideal, by allowing (time-consistent) policy to be set by a process of bargaining between two political players with different weights on elements of the social welfare function. We characterize the (linear) Markov perfect equilibrium and, in a series of numerical examples, we explore the resultant policy response to shocks which cannot be perfectly offset with the available instruments due to their fiscal consequences. We find that, even although the players, on average, have the socially desirable objective function, the process of bargaining implies an outcome which deviates from the time-consistent policy chosen by the benevolent policy maker. Moreover, the range of instruments available mean that policy makers will bargain across the entire policy mix, sometimes implying outcomes which are quite different from those that would be chosen by a single policy maker. These policy outcomes depend crucially on the nature of the conflict and also the level of government debt. [ABSTRACT FROM AUTHOR]

The game of resource extraction/capital accumulation is a stochastic infinite-horizon game, which models a joint utilization of a productive asset over time. The paper complements the available results on pure Markov perfect equilibrium existence in the non-symmetric game setting with an arbitrary number of agents. Moreover, we allow that the players have unbounded utilities and relax the assumption that the stochastic kernels of the transition probability must depend only on the amount of resource before consumption. This class of the game has not been examined beforehand. However, we could prove the Markov perfect equilibrium existence only in the specific case of interest. Namely, when the players have constant relative risk aversion (CRRA) power utilities and the transition law follows a geometric random walk in relation to the joint investment. The setup with the chosen characteristics is motivated by economic considerations, which makes it relevant to a certain range of real-word problems.

Managers, let stockpiling be but at a higher price—don't hope to cut stockpiling by lopping off promotions. One goal of promotions for frequently purchased products is increasing short-term sales. Increases could be at competitors' expense, coming from consumers with relatively weak brand preferences. However, increased sales from brand-loyal consumers could well cannibalize future sales of the promoted brand. An unintended consequence of promotions is that loyal consumers otherwise willing to pay high prices may strategically stockpile at low prices. What is its impact on firms' profits? Who benefits from stockpiling? How should firms adapt their pricing to accommodate consumer stockpiling? For answers, we analyze an infinite horizon dynamic model of competition and derive the Markov perfect equilibrium pricing strategies that yield several managerial insights. We find strategic stockpiling does not reduce firms' long-run profits when managers adopt pricing strategies we identify. Turning to strategies, stockpiling causes firms to move away from frequently promoting below the stockpiling threshold, but it leads to mass points at reservation price and stockpiling threshold. Stockpiling is beneficial to consumers who stockpile but hurts those that do not stockpile, whereas switchers remain largely unaffected. State-dependent pricing as a result of stockpiling leads to positive intertemporal price correlation, implying, counterintuitively, that in equilibrium, deep promotions should be followed by deep promotions. [ABSTRACT FROM AUTHOR]

We study Markov decision processes with Borel state spaces under quasi-hyperbolic discounting. This type of discounting nicely models human behaviour, which is time-inconsistent in the long run. The decision maker has preferences changing in time. Therefore, the standard approach based on the Bellman optimality principle fails. Within a dynamic game-theoretic framework, we prove the existence of randomised stationary Markov perfect equilibria for a large class of Markov decision processes with transitions having a density function. We also show that randomisation can be restricted to two actions in every state of the process. Moreover, we prove that under some conditions, this equilibrium can be replaced by a deterministic one. For models with countable state spaces, we establish the existence of deterministic Markov perfect equilibria. Many examples are given to illustrate our results, including a portfolio selection model with quasi-hyperbolic discounting. [ABSTRACT FROM AUTHOR]

In many domains, committees bargain over a sequence of policies and a policy remains in effect until a new agreement is reached. In this paper, I argue that, in order to assess the consequences of veto power, it is important to take into account this dynamic aspect. I analyze an infinitely repeated divide-the-dollar game with an endogenous status quo policy. I show that full appropriation by the veto player is the only stable policy when legislators are sufficiently impatient; and that, irrespective of legislators' patience and the initial division of resources, there is always an equilibrium where policy eventually gets arbitrarily close to full appropriation by the veto player. In this equilibrium, increasing legislators' patience or decreasing the veto player's proposal power makes convergence to this outcome slower and the veto player supports reforms that decrease his allocation. The main predictions of the theory find support in controlled laboratory experiments. [ABSTRACT FROM AUTHOR]

This paper studies optimal growth strategies of a multiproduct firm that invests in the qualities of different products, which have persistent effects on future payoffs and are modeled as a state variable of a stochastic game. We derive a unique Markov perfect equilibrium under a monotonicity condition. At the early stage, the firm focuses on the product with higher quality, and may switch its specialization. If the quality of the specialized good is high enough, the firm diversifies to capture demands for all products. However, the firm may lose its focus on either product and get no demand, due to a moral hazard problem. [ABSTRACT FROM AUTHOR]

This paper examines competition between two main retailers, Wal-Mart and Target, in the context of expansion strategies. I use a structural model assuming that the firms are forward-looking and engage in dynamic strategies forming a Markov perfect equilibrium (MPE). The results show different expansion strategies and the asymmetric impact of the competitor's presence on each retailer. Target must operate in a comfort zone to benefit from the spillover effects of Wal-Mart's presence and avoid cannibalizing its own stores. On the other hand, Wal-Mart dominates most market structures and is not impacted significantly by Target. This result can be generalized to the case where small retailers benefit from clustering with their own stores and competitors' stores. An implication is that competing retailers engage in back-and-forth expansion strategies as one tries to penetrate a market and the other tries to defend. However, unlimited expansion results in cannibalization. The fixed cost for Target also contains a comfort zone due to a quadratic relationship with the number of stores owned, but the relationship is monotonically increasing for Wal-Mart. Another notable result is that Target's fixed cost per store added can be as high as nine times that of Wal-Mart. Overall, the results show that primary and secondary players can co-exist in the same market to reap benefits despite having different scales of operations. The results provide practical implications for policymakers who can use these insights to design retail and zoning regulations that protect local shops and maintain fair competition. [ABSTRACT FROM AUTHOR]

We analyze undiscounted continuous-time games of strategic experimentation with two-armed bandits. The risky arm generates payoffs according to a Lévy process with an unknown average payoff per unit of time which nature draws from an arbitrary finite set. Observing all actions and realized payoffs, plus a free background signal, players use Markov strategies with the common posterior belief about the unknown parameter as the state variable. We show that the unique symmetric Markov perfect equilibrium can be computed in a simple closed form involving only the payoff of the safe arm, the expected current payoff of the risky arm, and the expected full-information payoff, given the current belief. In particular, the equilibrium does not depend on the precise specification of the payoff-generating processes. [ABSTRACT FROM AUTHOR]

This paper examines the execution problems of large traders with a generalized price impact. Constructing two related models in a discrete-time setting, we solve these problems by applying the backward induction method of dynamic programming. In the first problem, we formulate the expected utility maximization problem of a single large trader as a Markov decision process and derive an optimal execution strategy. Then, in the second model, we formulate the expected utility maximization problem of two large traders as a Markov game and derive an equilibrium execution strategy at a Markov perfect equilibrium. Both of these two models enable us to investigate how the execution strategies and trade performances of a large trader are affected by the existence of other traders. Moreover, we find that these optimal and equilibrium execution strategies become deterministic when the total execution volumes of non-large traders are deterministic. We also show, by some numerical examples, the comparative statics results with respect to several problem parameters. [ABSTRACT FROM AUTHOR]

This paper analyzes how the transferability of production capacities from an established to a new product influences the incentives of a firm to invest in R&D. A dynamic duopoly model is considered, where initially both firms offer a homogeneous product. The firms invest in production capacities and simultaneously in R&D which determines their innovation rate. The firm that innovates first extends its product line and obtains a patent for the new product that prevents the other firm from catching up. Upon the launch of the new product, the innovator then has the option to transfer part of the capacity for the established product to the production process of the new product. If capacities can be rolled over to the new product, a trade-off can be detected in that this rollover option gives the larger firm more incentive to innovate, whereas the cannibalization effect gives the smaller firm a higher innovation incentive. As a logical consequence we find that the larger firm is expected to innovate first when the capacity transfer does not involve a too high capacity loss. However, if the losses of capacity transfer are considerable, the cannibalization effect starts to dominate and the smaller firm's incentive to innovate is larger. [ABSTRACT FROM AUTHOR]

We study a discrete-time non-stationary decision model in which the preferences of the decision maker change over time and are described by quasi-hyperbolic discounting. A time-consistent optimal solution in this model corresponds with a Markov perfect equilibrium in a stochastic game with uncountable state space played by countably many short-lived players. We show that Markov perfect equilibria may be constructed using a generalized policy iteration algorithm. This method is in part inspired by the fundamental works of Mertens and Parthasarathy (in: Raghavan, Ferguson, Parthasarathy, Vrieze (eds) Stochastic games and related topics, Kluwer Academic Publishers, Dordrecht, 1991; in: Neyman, Sorin (eds) Stochastic games and applications, Academic Publishers, Dordrecht, 2003) devoted to subgame perfect equilibria in standard n-person discounted stochastic games. If the one-period utilities and transition probabilities are independent of time, we obtain on new existence results on stationary Markov perfect equilibria in the models with unbounded from above utilities. [ABSTRACT FROM AUTHOR]

In this paper, we consider a stochastic economic growth model in the form of an intergenerational dynamic game. Both paternalistic and non-paternalistic components are present in the model. Under very general assumptions allowing for unbounded utility functions and weakly continuous transitions, we establish the existence of Markov perfect equilibria that consist of a consumption strategy and an indirect utility function. In the pure paternalistic case, we obtain new results on equilibria. An important point of our contribution is that we make no separability assumptions on the utility functions of generations. [ABSTRACT FROM AUTHOR]

In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm. [ABSTRACT FROM AUTHOR]

We study the pricing policy optimization problem for cloud providers while considering three properties of the real-world market: 1) providers have only incomplete information about the market; 2) it is in evolution due to the increasing number of users and decreasing marginal cost of providers; 3) it is fully competitive because of providers' and users' revenuedriven nature. As far as we know, there is no existing work investigating the optimal pricing policies under such realistic settings. We first propose a comprehensive model for the real-world cloud market and formulate it as a stochastic game. Then we use the Markov perfect equilibrium (MPE) to describe providers' optimal policies. Next we decompose the problem of computing the MPE into two subtasks: 1) dividing the stochastic game into many normal-formal games and calculating their Nash equilibria, for which we develop an algorithm ensuring to converge, and 2) computing the MPE of the original game, which is efficiently solved by an algorithm combining the Nash equilibria based on a mild assumption. Experimental results show that our algorithms are efficient for computing MPE and the MPE strategy leads to much higher profits for providers compared with existing policies. [ABSTRACT FROM AUTHOR]

Internet services, such as review sites, FAQ sites, online auction sites, online flea markets, and social networking services, are essential to our daily lives. Each Internet service aims to promote information exchange among people who share common interests, activities, or goods. Internet service providers aim to have users of their services actively communicate through their services. Without active interaction, the service falls into disuse. In this study, we consider that an Internet service has a network externality as its main feature, and we model user behavior in the Internet service with network externality (ISNE) as a dynamic game. In particular, we model the diffusion process of users of an ISNE as an infinite-horizon extensive-form game of complete information in which: (1) each user can choose whether or not to use the ISNE in her/his turn and (2) the network effect of the ISNE depends on the history of each player's actions. We then apply Markov perfect equilibrium to analyze how to increase the number of active users. We derive the necessary and sufficient condition under which the state in which every player is an active user is the unique Markov perfect equilibrium outcome. Moreover, we propose an incentive mechanism that enables the number of active users to increase steadily. [ABSTRACT FROM AUTHOR]

We reformulate the quality ladder model of Pakes and McGuire, Rand Journal of Economics, 25(4), 555-589 (1994) as a dynamic stochastic game with random moves in which each period one firm is picked at random to make an investment decision. Contrasting this model to the standard version with simultaneous moves illustrates the computational advantages of random moves. In particular, the quality ladder model with random moves avoids the curse of dimensionality in computing firms' expectations over all possible future states and is therefore orders of magnitude faster to solve than its counterpart with simultaneous moves when there are more than just a few firms. Perhaps unexpectedly, the equilibria of the quality ladder model with random moves are practically indistinguishable from those of the model with simultaneous moves. [ABSTRACT FROM AUTHOR]

Morton and Wecker (1977) stated that the value iteration algorithm solves a dynamic program's policy function faster than its value function when the limiting Markov chain is ergodic. I show that their proof is incomplete, and provide a new proof of this classic result. I use this result to accelerate the estimation of Markov decision processes and the solution of Markov perfect equilibria. [ABSTRACT FROM AUTHOR]

We study consumer strategy, vendor strategy and their equilibrium in a duopoly market where vendors bear the costs in the production process. If consumers have an identical preference that is unknown, we obtain the equations satisfied by the total social surplus and the profit of each vendor. Moreover, we prove that the second derivatives of the total social surplus function and the profit function are continuous at the cutoff by the well-known conditions of value matching and smooth pasting. In a duopoly market, vendors can increase the total social surplus by adjusting their output, which causes fewer profits for vendors. However, rational vendors usually do not make such concessions, which implies that it is almost impossible to maximize total social surplus in a duopoly market. Thus, the planner needs to control vendors' output to maximize the total social surplus. We obtain the planner's optimal strategy and the socially efficient allocation. When vendors and consumers maximize their value, we find that the price of goods may be less than their costs due to price competition. By Bayesian learning and price competition model, we discuss a Markov perfect equilibrium and the relationship between the socially efficient allocation and the Markov perfect equilibrium. [ABSTRACT FROM AUTHOR]

The paper examines the issue of international pollution control under a dynamic framework involving heterogeneous countries and a critical threshold. We propose an appealing specification for this type of problem, and its novelty is illustrated by the fact that countries use linear strategies in equilibrium, although their utility and damage functions are highly convex. The stock of pollution under cooperation is always lower. Surprisingly, emissions are always higher in distant periods under the cooperative scenario; and we show through examples that this is a significant finding rather than some marginal effect. It is also shown that the efficient allocation of pollution abatement is driven only by the respective technologies of the countries. Finally, a dynamic transfer scheme allowing to implement the cooperative solution is proposed. [ABSTRACT FROM AUTHOR]

In this note, we prove the existence of a Markov perfect equilibrium in a non-stationary version of a paternalistic bequest game. The method we advocate is general and allows to study models with unbounded state space and unbounded utility functions. We cover both, the stochastic and deterministic cases. We provide a characterization of the set of all Markov perfect equilibria by means of a set-valued recursive equation involving the best response operator. In the stationary case, we show that there exists a set of strategies that is invariant under the best response mapping. [ABSTRACT FROM AUTHOR]