Search

Autonomous vehicles will be an integral part of ride-sharing services in the future. This setting differs from traditional ride-sharing marketplaces because of the absence of the supply side (drivers). However, it has far-reaching consequences because in addition to pricing, players now have to make decisions on how to distribute fleets across network locations and re-balance vehicles in order to serve future demand. In this paper, we explore a duopoly setting in the ride-sharing marketplace where the players have fully autonomous fleets. Each ride-service provider (RSP)'s prices depend on the prices and the supply of the other player. We formulate their decision-making problems using a game-theoretic setup where each player seeks to find the optimal prices and supplies at each node while considering the decisions of the other player. This leads to a scenario where the players' optimization problems are coupled and it is challenging to find the equilibrium. We characterize the types of demand functions (e.g.: linear) for which this game admits an exact potential function and can be solved efficiently. For other types of demand functions, we propose an iterative algorithm to compute the equilibrium. We conclude by providing numerical insights into how different kinds of equilibria would play out in the market when the players are asymmetric. Our numerical evaluations also provide insights into how the regulator needs to consider network effects while deciding regulation in order to avoid unfavorable outcomes., Comment: Submitted to MobiHoc'23

We consider a multi-agent episodic MDP setup where an agent (leader) takes action at each step of the episode followed by another agent (follower). The state evolution and rewards depend on the joint action pair of the leader and the follower. Such type of interactions can find applications in many domains such as smart grids, mechanism design, security, and policymaking. We are interested in how to learn policies for both the players with provable performance guarantee under a bandit feedback setting. We focus on a setup where both the leader and followers are {\em non-myopic}, i.e., they both seek to maximize their rewards over the entire episode and consider a linear MDP which can model continuous state-space which is very common in many RL applications. We propose a {\em model-free} RL algorithm and show that $\tilde{\mathcal{O}}(\sqrt{d^3H^3T})$ regret bounds can be achieved for both the leader and the follower, where $d$ is the dimension of the feature mapping, $H$ is the length of the episode, and $T$ is the total number of steps under the bandit feedback information setup. Thus, our result holds even when the number of states becomes infinite. The algorithm relies on {\em novel} adaptation of the LSVI-UCB algorithm. Specifically, we replace the standard greedy policy (as the best response) with the soft-max policy for both the leader and the follower. This turns out to be key in establishing uniform concentration bound for the value functions. To the best of our knowledge, this is the first sub-linear regret bound guarantee for the Markov games with non-myopic followers with function approximation., Comment: arXiv admin note: text overlap with arXiv:2206.11889

Autonomous vehicles (AVs) are attractive for ride service providers (RSPs) in part because they eliminate the need to compete for human drivers. We investigate a scenario where two RSPs with AVs compete for customers. We model the problem as a game where the RSPs select prices for each origin-destination pair over multiple time periods in an underlying graph representing the customers' desired trips. Each RSP also decides the number of AVs to be stationed at each node at each time period to serve the customers' demands. The number of customers who avail of the service of an RSP depends on the price selected by the RSP and its competitor. Since the strategy choices available to an RSP depends on its competitor, we seek to compute a Generalized Nash equilibrium (GNE). We show that there may be multiple GNEs. However, when an RSP selects prices in order to deter its competitor when it is not serving a source-destination pair, the game has a potential function and admits a unique GNE. We also compare the competitive prices with a monopoly price where only one RSP is in the market. Numerically, we show that if a network consists of two equal-size spatial clusters of demand where the demand between clusters is low, the RSPs may partition the market, i.e, one cluster is served by only one RSP. Hence, the competitive price may become close to the monopoly price., Comment: Submitted in IEEE Conference on Decision and Control

We consider a multi-agent Markov strategic interaction over an infinite horizon where agents can be of multiple types. We model the strategic interaction as a mean-field game in the asymptotic limit when the number of agents of each type becomes infinite. Each agent has a private state; the state evolves depending on the distribution of the state of the agents of different types and the action of the agent. Each agent wants to maximize the discounted sum of rewards over the infinite horizon which depends on the state of the agent and the distribution of the state of the leaders and followers. We seek to characterize and compute a stationary multi-type Mean field equilibrium (MMFE) in the above game. We characterize the conditions under which a stationary MMFE exists. Finally, we propose Reinforcement learning (RL) based algorithm using policy gradient approach to find the stationary MMFE when the agents are unaware of the dynamics. We, numerically, evaluate how such kind of interaction can model the cyber attacks among defenders and adversaries, and show how RL based algorithm can converge to an equilibrium., Comment: 9 pages

We consider a scenario where a retailer can set different prices for different consumers in a smart grid. The retailer's objective is to maximize the revenue, minimize the operating cost, and maximize the consumer's welfare. The retailer wants to optimize a convex combination of the above objectives using price signals specific to each consumer. However, variability in unit prices across consumers is bounded by a parameter $\eta$, hence limiting the discrimination. We formulate the pricing problem as a Stackelberg game where the retailer is the leader and consumers are followers. Since the retailer's optimization problem turns out to be non-convex, we convexify it via relaxations. We provide performance guarantees for the relaxations in the asymptotic sense (when number of consumers tends to $\infty$). Further, we show that despite the variability in pricing, the pricing scheme proposed by our model is fair as higher prices are charged to consumers who have higher willingness for demand. We extend our analysis to the scenario where consumers can feed energy back to the grid via net-metering. We show that our pricing policy promotes fairness even in this scenario as prosumers who contribute more to the grid, are given large cuts on buying rates. The policy is also found to incentivize more prosumers to invest in renewable energy, thus encouraging sustainability., Comment: Under review

The Citizens Broadband Radio Service (CBRS) recently adopted in the U.S. enables two tiers of commercial users to share spectrum with a third tier of incumbent users. This sharing can be further assisted by Environmental Sensing Capability operators (ESCs), that monitor the spectrum occupancy to determine when use of the spectrum will not harm incumbents. Two key aspects of this framework that impact how firms may compete are the differences in information provided by different ESCs and the different tiers in which a firm may access the spectrum. We develop a game theoretic model that captures both of these features and analyze it to gain insight into their impact. Specifically, we consider a priority access (PA) tier firm has access to the both licensed band and unlicensed band, and a general authorized access (GAA) tier firm has access only to the unlicensed band. The PA tier and GAA tier firms compete for users. Our analysis reveals that the amount of unlicensed and licensed bandwidth in the CBRS must be chosen judiciously in order to maximize the social welfare. We also show that a limited amount of unlicensed access of PA tier firm is beneficial to the user's surplus as well as to the social welfare., Comment: Submitted to MobiHoc'2019

We consider users which may have renewable energy harvesting devices, or distributed generators. Such users can behave as consumer or producer (hence, we denote them as prosumers) at different time instances. A prosumer may sell the energy to other prosumers in exchange of money. We consider a demand response model, where the price of conventional energy depends on the total demand of all the prosumers at a certain time. A prosumer depending on its own utility has to select the amount of energy it wants to buy either from the grid or from other prosumers, or the amount of excess energy it wants to sell to other prosumers. However, the strategy, and the payoff of a prosumer inherently depends on the strategy of other prosumers as a prosumer can only buy if the other prosumers are willing to sell. We formulate the problem as a coupled constrained game, and seek to obtain the generalized Nash equilibrium. We show that the game is a concave potential game and show that there exists a unique generalized Nash equilibrium. We consider that a platform will set the price for distributed interchange of energy among the prosumers in order to minimize the consumption of the conventional energy. We propose a distributed algorithm where the platform sets a price to each prosumer, and then each prosumer at a certain time only optimizes its own payoff. The prosumer then updates the price depending on the supply and demand for each prosumer. We show that the algorithm converges to an optimal generalized Nash equilibrium. The distributed algorithm also provides an optimal price for the exchange market., Comment: 9 pages, 4 figures

In cloud storage, the digital data is stored in logical storage pools, backed by heterogeneous physical storage media and computing infrastructure that are managed by a Cloud Service Provider (CSP). To balance the tradeoff between service performance and cost, CSPs often employ different storage tiers, for instance, cold storage and hot storage. Storing data in hot storage incurs high storage cost yet delivers low access latency, whereas cold storage is able to inexpensively store massive amounts of data and thus provides lower cost with higher latency. In this paper, we address a major challenge confronting the CSPs utilizing such tiered storage architecture - how to maximize their overall profit over a variety of storage tiers that offer distinct characteristics, as well as file placement and access request scheduling policies. To this end, we propose a scheme where the CSP offers a two-stage auction process for (a) requesting storage capacity, and (b) requesting accesses with latency requirements. Our two-stage bidding scheme provides a hybrid storage and access optimization framework with the objective of maximizing the CSP's total net profit over four dimensions: file acceptance decision, placement of accepted files, file access decision and access request scheduling policy. The proposed optimization is a mixed-integer nonlinear program that is hard to solve. We propose an efficient heuristic to relax the integer optimization and to solve the resulting nonlinear stochastic programs. The algorithm is evaluated under different scenarios and with different storage system parameters, and insightful numerical results are reported by comparing the proposed approach with other profit-maximization models. We see a profit increase of over 60% of our proposed method compared to other schemes in certain simulation scenarios., Comment: 14 pages

The paper considers a bidirectional power flow model of the electric vehicles (EVs) in a charging station. The EVs can inject energies by discharging via a Vehicle-to-Grid (V2G) service which can enhance the profits of the charging station. However, frequent charging and discharging degrade battery life. A proper compensation needs to be paid to the users to participate in the V2G service. We propose a menu-based pricing scheme, where the charging station selects a price for each arriving user for the amount of battery utilization, the total energy, and the time (deadline) that the EV will stay. The user can accept one of the contracts or rejects all depending on their utilities. The charging station can serve users using a combination of the renewable energy and the conventional energy bought from the grid. We show that though there exists a profit maximizing price which maximizes the social welfare, it provides no surplus to the users if the charging station is aware of the utilities of the users. If the charging station is not aware of the exact utilities, the social welfare maximizing price may not maximize the expected profit. In fact, it can give a zero profit. We propose a pricing strategy which provides a guaranteed fixed profit to the charging station and it also maximizes the expected profit for a wide range of utility functions. Our analysis shows that when the harvested renewable energy is small the users have higher incentives for the V2G service. We, numerically, show that the charging station's profit and the user's surplus both increase as V2G service is efficiently utilized by the pricing mechanism., Comment: Submitted in IEEE Transactions on Automatic Control. arXiv admin note: substantial text overlap with arXiv:1609.09037

We propose an online pricing mechanism for electric vehicle (EV) charging. A charging station decides prices for each arriving EV depending on the energy and the time within which the EV will be served (i.e. deadline). The user selects either one of the contracts by paying the prescribed price or rejects all depending on their surpluses. The charging station can serve users using renewable energy and conventional energy. Users may select longer deadlines as they may have to pay less because of the less amount of conventional energy, however, they have to wait a longer period. We consider a {\em myopic} charging station and show that there exists a pricing mechanism which jointly maximizes the social welfare and the profit of the charging station when the charging station knows the utilities of the users. However, when the charging station does not know the utilities of the users, the social welfare pricing strategy may not maximize the expected profit of the charging station and even the profit may be $0$. We propose a fixed profit pricing strategy which provides a guaranteed fixed profit to the charging station and can maximize the profit in practice. We empirically show that our proposed mechanism reduces the peak-demand and utilizes the limited charging spots in a charging station efficiently., Comment: 31 pages, 11 figures, Accepted to IEEE Transactions on Smart Grid

In a secondary spectrum market primaries set prices for their unused channels to the secondaries. The payoff of a primary depends on the availability of unused channels of its competitors. We consider a model were a primary can acquire its competitor's channel state information (C-CSI) at a cost. We formulate a game between two primaries where each primary decides whether to acquire C-CSI or not and then selects its price based on that. We first characterize the Nash Equilibrium (NE) of this game for a symmetric model where the C-CSI is perfect. We show that the payoff of a primary is independent of the C-CSI acquisition cost. We then generalize our analysis to allow for imperfect estimation and cases where the two primaries have different C-CSI costs or different channel availabilities. Our results show interestingly that the payoff of a primary increases when there is estimation error. We also show that surprisingly, the expected payoff of a primary may decrease when the C-CSI acquisition cost decreases when primaries have different availabilities., Comment: Shorter version will be presented in ISIT'16. Submitted to JSAC Special Issue on Spectrum Sharing, 2016. Impacts of Estimation errors, different C-CSI acquisition costs and different availabilities are now considered

The economic model of the Internet of Things (IoT) consists of end users, advertisers and three different kinds of providers--IoT service provider (IoTSP), Wireless service provider (WSP) and cloud service provider (CSP). We investigate three different kinds of interactions among the providers. First, we consider that the IoTSP prices a bundled service to the end-users, and the WSP and CSP pay the IoTSP (push model). Next, we consider the model where the end-users independently pay the each provider (pull model). Finally, we consider a hybrid model of the above two where the IoTSP and WSP quote their prices to the end-users, but the CSP quotes its price to the IoTSP. We characterize and quantify the impact of the advertisement revenue on the equilibrium pricing strategy and payoff of providers, and corresponding demands of end users in each of the above interaction models. Our analysis reveals that the demand of end-users, and the payoffs of the providers are non decreasing functions of the advertisement revenue. For sufficiently high advertisement revenue, the IoTSP will offer its service free of cost in each interaction model. However, the payoffs of the providers, and the demand of end-users vary across different interaction models. Our analysis shows that the demand of end-users, and the payoff of the WSP are the highest in the pull (push, resp.) model in the low (high, resp.) advertisement revenue regime. The payoff of the IoTSP is always higher in the pull model irrespective of the advertisement revenue. The payoff of the CSP is the highest in the hybrid model in the low advertisement revenue regime. However, in the high advertisement revenue regime the payoff of the CSP in the hybrid model or in the push model can be higher depending on the equilibrium chosen in the push model., Comment: Submitted in IEEE Transactions on Automatic Control

We investigate a spectrum oligopoly market where each primary seeks to sell secondary access to its channel at multiple locations. Transmission qualities of a channel evolve randomly. Each primary needs to select a price and a set of non-interfering locations (which is an independent set in the conflict graph of the region) at which to offer its channel without knowing the transmission qualities of the channels of its competitors. We formulate the above problem as a non-cooperative game. We consider two scenarios-i) when the region is small, ii) when the region is large. In the first setting, we focus on a class of conflict graphs, known as mean valid graphs which commonly arise when the region is small. We explicitly compute a symmetric Nash equilibrium (NE); the NE is threshold type in that primaries only choose independent set whose cardinality is greater than a certain threshold. The threshold on the cardinality increases with increase in quality of the channel on sale. We show that the symmetric NE strategy profile is unique in a special class of conflict graphs (linear graph). In the second setting, we consider node symmetric conflict graphs which arises when the number of locations is large (potentially, infinite). We explicitly compute a symmetric NE that randomizes equally among the maximum independent sets at a given channel state vector. In the NE a primary only selects the maximum independent set at a given channel state vector. We show that the two symmetric NEs computed in two settings exhibit important structural difference., Comment: Added several results from the other versions. Submitted to IEEE Transactions on Information Theory. 63 pages one column, 22 figures

We investigate a spectrum oligopoly market where primaries lease their channels to secondaries in lieu of financial remuneration. Transmission quality of a channel evolves randomly. Each primary has to select the price it would quote without knowing the transmission qualities of its competitors' channels. Each secondary buys a channel depending on the price and the transmission quality a channel offers. We formulate the price selection problem as a non co-operative game with primaries as players. In the one-shot game, we show that there exists a unique symmetric Nash Equilibrium(NE) strategy profile and explicitly compute it. Our analysis reveals that under the NE strategy profile a primary prices its channel to render high quality channel more preferable to the secondary; this negates the popular belief that prices ought to be selected to render channels equally preferable to the secondary regardless of their qualities. We show the loss of revenue in the asymptotic limit due to the non co-operation of primaries. In the repeated version of the game, we characterize a subgame perfect NE where a primary can attain a payoff arbitrarily close to the payoff it would obtain when primaries co-operate., Comment: Accepted for publication in IEEE/ACM Transactions on Networking. 41 pages single column format.Conference version is available at arXiv:1305.3351

We investigate a spectrum oligopoly where primary users allow secondary access in lieu of financial remuneration. Transmission qualities of the licensed bands fluctuate randomly. Each primary needs to select the price of its channel with the knowledge of its own channel state but not that of its competitors. Secondaries choose among the channels available on sale based on their states and prices. We formulate the price selection as a non-cooperative game and prove that a symmetric Nash equilibrium (NE) strategy profile exists uniquely. We explicitly compute this strategy profile and analytically and numerically evaluate its efficiency. Our structural results provide certain key insights about the unique symmetric NE., Comment: Presented in ISIT' 2013, Istanbul Version 2 contains some modified versions of proofs of version 1. In IEEE Proceedings of International Symposium on Information Theory, 2013