Search

We consider the following distributed service model: jobs with unit mean, general distribution, and independent processing times arrive as a renewal process of rate $\lambda n$, with $0, Comment: 44 pages

We consider the following distributed service model: jobs with unit mean, exponentially distributed, and independent processing times arrive as a Poisson process of rate $\lambda n$, with $00$ (no matter how small), if our policy only uses a linear message rate $\alpha n$, the resulting asymptotic delay is upper bounded, uniformly over all $\lambda

We consider a Cournot oligopoly model where multiple suppliers (oligopolists) compete by choosing quantities. We compare the social welfare achieved at a Cournot equilibrium to the maximum possible, for the case where the inverse market demand function is convex. We establish a lower bound on the efficiency of Cournot equilibria in terms of a scalar parameter derived from the inverse demand function, namely, the ratio of the slope of the inverse demand function at the Cournot equilibrium to the average slope of the inverse demand function between the Cournot equilibrium and a social optimum. Also, for the case of a single, monopolistic, profit maximizing supplier, or of multiple suppliers who collude to maximize their total profit, we establish a similar but tighter lower bound on the efficiency of the resulting output. Our results provide nontrivial quantitative bounds on the loss of social welfare for several convex inverse demand functions that appear in the economics literature. © 2014 Elsevier B.V., National Science Foundation (Grant CMMI-0856063)

We compare the aggregate profit achieved at a Cournot equilibrium to the maximum possible, which would be obtained if the suppliers were to collude. We establish a lower bound on the profit of Cournot equilibria in terms of a scalar parameter that captures qualitative properties of the inverse demand function and the number of suppliers (or the maximum of the suppliers’ market shares). The lower bounds are tight when the inverse demand function is affine.

We derive lower bounds on the convergence speed of a widely used class of distributed averaging algorithms. In particular, we prove that any distributed averaging algorithm whose state consists of a single real number and whose (possibly nonlinear) update function satisfies a natural smoothness condition has a worst case running time of at least on the order of n2 on a line network of n nodes. Our results suggest that increased memory or expansion of the state space is crucial for improving the running times of distributed averaging algorithms.

We consider a model where a finite number of producers compete to meet an infinitely divisible but inelastic demand for a product. Each firm is characterized by a production cost that is convex in the output produced, and firms act as profit maximizers. We consider a uniform price market design that uses supply function bidding: firms declare the amount they would supply at any positive price, and a single price is chosen to clear the market. We are interested in evaluating the impact of price-anticipating behavior both on the allocative efficiency of the market and on the prices seen at equilibrium. We show that by restricting the strategy space of the firms to parameterized supply functions, we can provide upper bounds on both the inflation of aggregate cost at the Nash equilibrium relative to the socially optimal level, as well as the markup of the Nash equilibrium price above the competitive level: as long as N > 2 firms are competing, these quantities are both upper bounded by 1 + 1/(N − 2). This result holds even in the presence of asymmetric cost structure across firms. We also discuss several extensions, generalizations, and related issues., National Science Foundation (U.S.) (Graduate Research Fellowship), National Science Foundation (U.S.) (grant ECS-0312921)

We consider a multiarmed bandit problem where the expected reward of each arm is a linear function of an unknown scalar with a prior distribution. The objective is to choose a sequence of arms that maximizes the expected total (or discounted total) reward. We demonstrate the effectiveness of a greedy policy that takes advantage of the known statistical correlation structure among the arms. In the infinite horizon discounted reward setting, we show that the greedy and optimal policies eventually coincide, and both settle on the best arm. This is in contrast with the Incomplete Learning Theorem for the case of independent arms. In the total reward setting, we show that the cumulative Bayes risk after T periods under the greedy policy is at most O(logT), which is smaller than the lower bound of ?(log2 T) established by Lai for a general, but different, class of bandit problems. We also establish the tightness of our bounds. Theoretical and numerical results show that the performance of our policy scales independently of the number of arms.

We show the optimality of state-dependent echelon base-stock policies in uncapacitated serial inventory systems with Markov-modulated demand and Markov-modulated stochastic lead times in the absence of order crossing. Our results cover finite-time horizon problems as well as infinite-time horizon formulations, with either a discounted or an average cost criterion. We employ a novel approach, based on a decomposition of the problem into a series of single-unit single-customer problems that are essentially decoupled. Besides providing a simple proof technique, this approach also gives rise to efficient algorithms for the calculation of the base-stock levels.

The unichain condition requires that every policy in an MDP result in a single ergodic class, and guarantees that the optimal average cost is independent of the initial state. We show that checking whether the unichain condition fails to hold is an NP-complete problem. We conclude with a brief discussion of the merits of the more general weak accessibility condition.

We consider an SIS-type epidemic process that evolves on a known graph. We assume that a fixed curing budget can be allocated at each instant to the nodes of the graph, towards the objective of minimizing the expected extinction time of the epidemic. We provide a lower bound on the optimal expected extinction time as a function of the available budget, the epidemic parameters, the maximum degree, and the CutWidth of the graph. For graphs with large CutWidth (close to the largest possible), and under a budget which is sublinear in the number of nodes, our lower bound scales exponentially with the size of the graph., United States. Army Research Office (Grant W911NF-12-1-0509), National Science Foundation (U.S.) (Grant CMMI-1234062)

We seek to maximize the average data throughput of a single transmitter sending data over a fading channel to a single user class. The transmitter has a fixed amount of energy and a limited amount of time to send data. Given that the channel state determines the throughput obtained per unit of energy expended, the goal is to obtain a policy for scheduling transmissions that maximizes the expected data throughput. We develop a dynamic programming formulation that leads to an optimal transmission schedule, first where the present channel state is known just before transmission, and then to the case where the current channel state is unknown before transmission, but observed after transmission and evolves according to a Markov process. We then extend our approach to the problem of minimizing the expected energy required to send a fixed amount of data over a fading channel given deadline constraints.

We study a linear stochastic approximation algorithm that arises in the context of reinforcement learning. The algorithm employs a decreasing step-size, and is driven by Markov noise with time-varying statistics. We show that under suitable conditions, the algorithm can track the changes in the statistics of the Markov noise, as long as these changes are slower than the rate at which the step-size of the algorithm goes to zero.

This paper proposes a simulation-based algorithm for optimizing the average reward in a finite-state Markov reward process that depends on a set of parameters. As a special case, the method applies to Markov decision processes where optimization takes place within a parametrized set of policies. The algorithm relies on the regenerative structure of finite-state Markov processes, involves the simulation of a single sample path, and can be implemented online. A convergence result (with probability 1) is provided.

The lower and average spectral radii measure, respectively, the minimal and average growth rates of long products of matrices taken from a finite set. The logarithm of the average spectral radius is traditionally called the Lyapunov exponent. When one performs these products in the max-algebra, we obtain quantities that measure the performance of discrete event systems. We show that approximating the lower and average max-algebraic spectral radii is NP-hard.

The authors develop a theory characterizing optimal stopping times for discrete-time ergodic Markov processes with discounted rewards. The theory differs from prior work by its view of per-stage and terminal reward functions as elements of a certain Hilbert space. In addition to a streamlined analysis establishing existence and uniqueness of a solution to Bellman's equation, this approach provides an elegant framework for the study of approximate solutions. In particular, the authors propose a stochastic approximation algorithm that tunes weights of a linear combination of basis functions in order to approximate a value function. They prove that this algorithm converges (almost surely) and that the limit of convergence has some desirable properties. The utility of the approximation method is illustrated via a computational case study involving the pricing of a path dependent financial derivative security that gives rise to an optimal stopping problem with a 100-dimensional state space.

We discuss the temporal-difference learning algorithm, as applied to approximating the cost-to-go function of an infinite-horizon discounted Markov chain. The algorithm we analyze updates parameters of a linear function approximator online during a single endless trajectory of an irreducible aperiodic Markov chain with a finite or infinite state space. We present a proof of convergence (with probability one), a characterization of the limit of convergence, and a bound on the resulting approximation error. Furthermore, our analysis is based on a new line of reasoning that provides new intuition about the dynamics of temporal-difference learning. In addition to proving new and stronger positive results than those previously available, we identify the significance of online updating and potential hazards associated with the use of nonlinear function approximators. First, we prove that divergence may occur when updates are not based on trajectories of the Markov chain. This fact reconciles positive and negative results that have been discussed in the literature, regarding the soundness of temporal-difference learning. Second, we present an example illustrating the possibility of divergence when temporal difference learning is used in the presence of a nonlinear function approximator.

We analyze the computability and the complexity of various definitions of spectral radii for sets of matrices. We show that the joint and generalized spectral radii of two integer matrices are not approximable in polynomial time, and that two related quantities—the lower spectral radius and the largest Lyapunov exponent—are not algorithmically approximable.

We consider switched queueing networks with a mix of heavy-tailed (i.e., arrival processes with infinite variance) and exponential-type traffic and study the delay performance of the max-weight policy, known for its throughput optimality and asymptotic delay optimality properties. Our focus is on the impact of heavy-tailed traffic on exponential-type queues/flows, which may manifest itself in the form of subtle rate-dependent phenomena. We introduce a novel class of Lyapunov functions (piecewise linear and nonincreasing in the length of heavy-tailed queues), whose drift analysis provides exponentially decaying upper bounds to queue-length tail asymptotics despite the presence of heavy tails. To facilitate a drift analysis, we employ fluid approximations, proving that if a continuous and piecewise linear function is also a “Lyapunov function” for the fluid model, then the same function is a “Lyapunov function” for the original stochastic system. Furthermore, we use fluid approximations and renewal theory in order to prove delay instability results, i.e., infinite expected delays in steady state. We illustrate the benefits of the proposed approach in two ways: (i) analytically, by studying the delay stability regions of single-hop switched queueing networks with disjoint schedules, providing a precise characterization of these regions for certain queues and inner and outer bounds for the rest. As a side result, we prove monotonicity properties for the service rates of different schedules that, in turn, allow us to identify “critical configurations” toward which the state of the system is driven, and that determine to a large extent delay stability; (ii) computationally, through a bottleneck identification algorithm, which identifies (some) delay unstable queues/flows in complex switched queueing networks by solving the fluid model from certain initial conditions. Keywords: switched queueing networks; max-weight policy; heavy-tailed traffic; fluid approximations; piecewise linear Lyapunov functions, National Science Foundation (U.S.) (Grant CNS-1217048), National Science Foundation (U.S.) (Grant CMMI-1234062), United States. Office of Naval Research (Grant N00014-12-1-0064), United States. Army Research Office (Grant W911NF-08-1-0238)

We study the problem of statistical multiplexing of cell streams that have correlations at multiple time-scales. Each stream is modeled by a singularly perturbed Markov-modulated process with some state transitions occurring much less frequently than others. One motivation of this model comes from variable-rate compressed video, where the fast time-scale dynamics may correspond to correlations between adjacent frames, while the slow time-scale dynamics may correspond to correlations which in the same scene of a video sequence. We develop a set of large deviations results to estimate the buffer overflow probabilities in various asymptotic regimes in the buffer size, rare transition probabilities, and the number of streams. Using these results, we characterize the multiplexing gain in both the channel capacity and the buffering requirements and highlight the impact of the slow time-scale of the streams. >

A decentralized Bayesian hypothesis testing problem is considered. It is analytically demonstrated that for the known signal in the Gaussian noise binary hypothesis problem, when there are two sensors with statistically independent identically distributed Gaussian observations (conditioned on the true hypothesis), there is no loss in optimality in using the same decision rule at both sensors. Also, a multiple hypothesis problem is considered; some structure is analytically established for an optimal set of decision rules. >

Two sensors obtain data vectors x and y, respectively, and transmit real vectors m/spl I.oarr//sub 1/(x) and m/spl I.oarr//sub 2/(y), respectively, to a fusion center. The authors obtain tight lower bounds on the number of messages (the sum of the dimensions of m/spl I.oarr//sub 1/ and m/spl I.oarr//sub 2/) that have to be transmitted for the fusion center to be able to evaluate a given function f/spl I.oarr/(x,y). When the function f/spl I.oarr/ is linear, they show that these bounds are effectively computable. Certain decentralized estimation problems can be cast in the framework and are discussed in some detail. In particular, the authors consider the case where x and y are random variables representing noisy measurements and f/spl I.oarr/(x,y)=E[z|x,y], where z is a random variable to be estimated. Furthermore, it is established that a standard method for combining decentralized estimates of Gaussian random variables has nearly optimal communication requirements. >

We consider a consensus algorithm in which every node in a time-varying undirected connected graph assigns equal weight to each of its neighbors. Under the assumption that the degree of any given node is constant in time, we show that the algorithm achieves consensus within a given accuracy ∈ on n nodes in time O(n[superscript 3]ln(n=∈)). Because there is a direct relation between consensus algorithms in time-varying environments and inhomogeneous random walks, our result also translates into a general statement on such random walks. Moreover, we give simple proofs that the convergence time becomes exponentially large in the number of nodes n under slight relaxations of the above assumptions. We prove that exponential convergence time is possible for consensus algorithms on fixed directed graphs, and we use an example of Cao, Spielman, and Morse to give a simple argument that the same is possible if the constant degrees assumption is even slightly relaxed.

We consider finite horizon Markov decision processes under performance measures that involve both the mean and the variance of the cumulative reward. We show that either randomized or history-based policies can improve performance. We prove that the complexity of computing a policy that maximizes the mean reward under a variance constraint is NP-hard for some cases, and strongly NP-hard for others. We finally offer pseudopolynomial exact and approximation algorithms., National Science Foundation (U.S.) (CMMI-0856063)

We consider a flow-level model of a network operating under an $\alpha$-fair bandwidth sharing policy (with $\alpha>0$) proposed by Roberts and Massouli\'{e} [Telecomunication Systems 15 (2000) 185-201]. This is a probabilistic model that captures the long-term aspects of bandwidth sharing between users or flows in a communication network. We study the transient properties as well as the steady-state distribution of the model. In particular, for $\alpha\geq1$, we obtain bounds on the maximum number of flows in the network over a given time horizon, by means of a maximal inequality derived from the standard Lyapunov drift condition. As a corollary, we establish the full state space collapse property for all $\alpha\geq1$. For the steady-state distribution, we obtain explicit exponential tail bounds on the number of flows, for any $\alpha>0$, by relying on a norm-like Lyapunov function. As a corollary, we establish the validity of the diffusion approximation developed by Kang et al. [Ann. Appl. Probab. 19 (2009) 1719-1780], in steady state, for the case where $\alpha=1$ and under a local traffic condition., Comment: Published in at http://dx.doi.org/10.1214/12-AAP915 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

We examine shortest path problems in acyclic networks in which arc costs are known functions of certain environment variables at network nodes. Each of these variables evolves according to an independent Markov process. The vehicle can wait at a node (at a cost) in anticipation of more favorable arc costs. We first develop two recursive procedures for the individual arc case, one based on successive approximations, and the other on policy iteration. We also solve the same problem via parametric linear programming. We show that the optimal policy essentially classifies the state of the environment variable at a node into two categories: green states for which the optimal action is to immediately traverse the arc, and red states for which the optimal action is to wait. We then extend these concepts for the entire network by developing a dynamic programming procedure that solves the corresponding problem. The complexity of this method is shown to be O(n2K + nK3), where n is the number of network nodes and K is the number of Markov states at each node. We present examples and discuss possible research extensions.

We consider a proportional allocation mechanism that gives to each user an amount of a resource proportional to the user's bid. We study a corresponding Bayesian game in which each user has incomplete information on the state or type of the other users. We prove the existence of a Bayes-Nash equilibrium. Furthermore, under some mild assumptions, we establish asymptotic efficiency: we show that the per user social welfare achieved at any Bayes-Nash equilibrium is asymptotically equal to the maximum possible.

We consider the problem of allocating a fixed amount of an infinitely divisible resource among multiple competing, fully rational users. We study the efficiency guarantees that are possible when we restrict to mechanisms that satisfy certain scalability constraints motivated by large scale communication networks; in particular, we restrict attention to mechanisms where users are restricted to one-dimensional strategy spaces. We first study the efficiency guarantees possible when the mechanism is not allowed to price differen- tiate. We study the worst-case efficiency loss (ratio of the utility associated with a Nash equilibrium to the maximum possible utility), and show that the proportional allocation mechanism of Kelly (1997) minimizes the efficiency loss when users are price anticipating. We then turn our attention to mechanisms where price differentiation is permitted; using an adaptation of the Vickrey-Clarke-Groves class of mechanisms, we con- struct a class of mechanisms with one-dimensional strategy spaces where Nash equilibria are fully efficient. These mechanisms are shown to be fully efficient even in general convex environments, under reasonable assumptions. Our results highlight a fundamental insight in mechanism design: when the pricing flexibility available to the mechanism designer is limited, restricting the strategic flexibility of bidders may actually improve the efficiency guarantee., National Science Foundation, Army Research Office, DARPA - Next Generation Internet Initiative, National Science Foundation Graduate Research Fellowship

The problem of computing a fixed point of a nonexpansive function f is considered. Sufficient conditions are provided under which a parallel, partially asynchronous implementation of the iteration $x: = f(x)$ converges. These results are then applied to (i) quadratic programming subject to box constraints, (ii) strictly convex cost network flow optimization, (iii) an agreement and a Markov chain problem, (iv) neural network optimization, and (v) finding the least element of a polyhedral set determined by a weakly diagonally dominant, Leontief system. Finally, simulation results illustrating the attainable speedup and the effects of asynchronism are presented.

We provide an example proving that there exists no quadratic Lyapunov function for a certain class of linear agreement/consensus algorithms, a fact that had been numerically verified in [5]. We also briefly discuss sufficient conditions for the existence of such a Lyapunov function.

Motivated by resource allocation problems in communication networks as well as power systems, we consider the design of market mechanisms for such settings which are robust to gaming behavior by market participants. Recent results in this work are reviewed, including: (1) efficiency loss guarantees for a data rate allocation mechanism first proposed by Kelly, both when link capacities are fixed and when they are elastic; (2) characterization of mechanisms that minimize the efficiency loss, within a certain class of “simple” mechanisms; (3) extensions to general networks; and (4) mechanism design for supply function bidding in electric power systems.

We consider a resource allocation problem where individual users wish to send data across a network to maximize their utility, and a cost is incurred at each link that depends on the total rate sent through the link. It is known that as long as users do not anticipate the effect of their actions on prices, a simple proportional pricing mechanism can maximize the sum of users' utilities minus the cost (called aggregate surplus). Continuing previous efforts to quantify the effects of selfish behavior in network pricing mechanisms, we consider the possibility that users anticipate the effect of their actions on link prices. Under the assumption that the links' marginal cost functions are convex, we establish existence of a Nash equilibrium. We show that the aggregate surplus at a Nash equilibrium is no worse than a factor of 4*sqrt{2} - 5 times the optimal aggregate surplus; thus, the efficiency loss when users are selfish is no more than approximately 34%., Originally Laboratory for Information and Decision Systems (MIT) Publication 2605

We explore the properties of a congestion game where users of a congested resource anticipate the effect of their actions on the price of the resource. When users are sharing a single resource, we show existence and uniqueness of the Nash equilibrium, and establish that the aggregate utility received by the users is at least 3/4 of the maximum possible aggregate utility. These results form part of a growing literature on the "price of anarchy," i.e., the extent to which selfish behavior affects system efficiency.

We consider a resource allocation problem where individual users wish to send data across a single link to maximize their utility, and a cost is incurred dependent on the total flow sent on the link. It has been shown that as long as users do not anticipate the effect of their actions on prices, a simple proportional pricing mechanism can maximize the sum of users' utilities minus the cost (called social welfare). Continuing previous efforts to quantify the effects of selfish behavior in network pricing mechanisms, our research considers the possibility that users may anticipate the effect of their actions on the link price. Under the simple assumption that the link's marginal cost function is convex, we establish existence of a Nash equilibrium, and show that the system utility is no worse than a factor of 4/spl radic/2-5 times the optimal system utility; thus, the efficiency loss when users are selfish is no more than approximately 34%.

We consider discrete time, finite state space Markov reward processes which depend on a set of parameters. Previously, we proposed a simulation-based methodology to tune the parameters to optimize the average reward. The resulting algorithms converge with probability 1, but may have a high variance. Here, we propose two approaches to reduce the variance, which however introduce a new bias into the update direction. We report numerical results which indicate that the resulting algorithms are robust with respect to a small bias.

Provides some general results on the convergence of a class of stochastic approximation algorithms and their parallel and asynchronous variants. The author then uses these results to study the Q-learning algorithm, a reinforcement learning method for solving Markov decision problems, and establishes its convergence under conditions more general than previously available. >

Considers the problem of optimal variable-rate lossy compression of Markov sources when the encoded bit stream is buffered before entering a fixed-rate channel. The authors discuss how to minimize the average distortion under this constraint. >

We study the distribution of steady-state queue lengths in multiclass queueing networks under a stable policy. We propose a general methodology based on Lyapunov functions for the performance analysis of infinite state Markov chains and apply it specifically to Markovian multiclass queueing networks. We establish a deeper connection between stability and performance of such networks by showing that if there exist linear or piecewise linear Lyapunov functions that show stability, then these Lyapunov functions can be used to establish geometric-type lower and upper bounds on the tail probabilities, and thus bounds on the expectation of the queue lengths. As an example of our results, for a reentrant line queueing network with two processing stations operating under a work-conserving policy, we show that $\mathrm{E}[L] = O(\frac{1}{1 - \rho^*)^2})$, where $L$ is the total number of customers in the system, and $\rho^*$ is the maximal actual or virtual traffic intensity in the network. In a Markovian setting, this extends a recent result by Dai and Vande Vate, which states that a reentrant line queueing network with two stations is globally stable if $\rho^*

The purpose of the fringe tracking algorithms is to maintain lock on the target star after acquisition and to obtain the most accurate estimate possible of the scientific quantity (or quantities) of interest in the presence of dynamic disturbances to the spacecraft/interferometer ensemble. This study carries out an analysis of the performance and robustness achievable by four candidate estimation techniques when applied to an ultra-high-precision fringe tracking task (5 micro-arcsecond ultimate accuracy). The first class of fringe trackers studied include the Extended Kalman Filter. This class is followed by extensions to second and third order nonlinear filters developed by the authors. The higher order filters have expanded regions of convergence. Third, we consider the use of an invariant filter (IF) to estimate the angle between two target stars (using POINTS as a test case). The IF offers the advantage of improved robustness in the dynamical case, being in effect `invariant' to dynamics. Finally Discrete Bayes Algorithms make use of Bayes' decision rule to propagate the a posteriori distribution of the true parameter and take into account the discrete character of the Poisson photon arrival events. Variations of these algorithms, known as multiple hypotheses trackers, offer great promise for dim star tracking. An exploration of filter performance with respect to several parameters is carried out analytically and selected Monte Carlo simulations are carried out both to verify analytical predictions and to study performance.

We introduce a new method to investigate stability of work-conserving policies in multiclass queueing networks. The method decomposes feasible trajectories and uses linear programming to test stability. We show that this linear program is a necessary and sufficient condition for the stability of all work-conserving policies for multiclass fluid queueing networks with two stations. Furthermore, we find new sufficient conditions for the stability of multiclass queueing networks involving any number of stations and conjecture that these conditions are also necessary. Previous research had identified sufficient conditions through the use of a particular class of (piecewise linear convex) Lyapunov functions. Using linear programming duality, we show that for two-station systems the Lyapunov function approach is equivalent to ours and therefore characterizes stability exactly.

We consider “approximate stationary” Markov chains in which the entries of the one-step transition probability matrix are known to be of different orders of magnitude and whose structure (that is, the orders of magnitude of the transition probabilities) does not change with time. For such Markov chains we present a method for generating order of magnitude estimates for the t-step transition probabilities, for any t. We then notice that algorithms of the simulated annealing type may be represented by Markov chains which are approximately stationary over fairly long time intervals. Using our results we obtain a characterization of the convergent “cooling” schedules for the most general class of algorithms of the simulated annealing type.

The complexity of two problems of distributed computation and decision-making is studied. It is shown that deciding whether two distant agents can arrive at compatible decisions without any communication can be done in polynomial time if there are two possible decisions for each agent, but is NP-complete if one agent has three or more alternatives. It is also shown that minimizing the amount of communication necessary for the distributed computation of a function, when two distant computers receive each a part of the input, is NP-complete. This proves a conjecture due to A. Yao.