Search

Markov decision processes (MDPs) are used to model stochastic systems in many applications. Several efficient algorithms to compute optimal policies have been studied in the literature, including value iteration (VI) and policy iteration. However, these do not scale well especially when the discount factor for the infinite horizon discounted reward, $\lambda$, gets close to $1$. In particular, the running time scales as $O \left( 1/(1-\lambda) \right)$ for these algorithms. In this paper, our goal is to design new algorithms that scale better than previous approaches when $\lambda$ approaches $1$. Our main contribution is to present a connection between VI and gradient descent and adapt the ideas of acceleration and momentum in convex optimization to design faster algorithms for MDPs. We prove theoretical guarantees of a faster convergence of our algorithms for the computation of the value function of a policy, where the running times of our algorithms scale as $O \left( 1/\sqrt{1-\lambda} \right)$ for reversible MDP instances. The improvement is quite analogous to Nesterov's acceleration and momentum in convex optimization. We also provide a lower bound on the convergence properties of any first-order algorithm for solving MDPs, presenting a family of MDPs instances for which no algorithm can converge faster than VI when the number of iterations is smaller than the number of states. We introduce a Safe Accelerated Value Iteration (S-AVI), which alternates between accelerated updates and value iteration updates. Our algorithm S-AVI is worst-case optimal and retains the theoretical convergence properties of VI while exhibiting strong empirical performances, providing significant speedups compared to classical approaches (up to one order of magnitude in many cases) for a large test bed of MDP instances.

We study a seemingly unexpected and relatively less understood overfitting aspect of a fundamental tool in sparse linear modeling - best subset selection, which minimizes the residual sum of squares subject to a constraint on the number of nonzero coefficients. While the best subset selection procedure is often perceived as the "gold standard" in sparse learning when the signal to noise ratio (SNR) is high, its predictive performance deteriorates when the SNR is low. In particular, it is outperformed by continuous shrinkage methods, such as ridge regression and the Lasso. We investigate the behavior of best subset selection in the high-noise regimes and propose an alternative approach based on a regularized version of the least-squares criterion. Our proposed estimators (a) mitigate, to a large extent, the poor predictive performance of best subset selection in the high-noise regimes; and (b) perform favorably, while generally delivering substantially sparser models, relative to the best predictive models available via ridge regression and the Lasso. We conduct an extensive theoretical analysis of the predictive properties of the proposed approach and provide justification for its superior predictive performance relative to best subset selection when the noise-level is high. Our estimators can be expressed as solutions to mixed integer second order conic optimization problems and, hence, are amenable to modern computational tools from mathematical optimization.

The motion-and-time analysis has been a popular research topic in operations research, especially for analyzing work performances in manufacturing and service operations. It is regaining attention as continuous improvement tools for lean manufacturing and smart factory. This paper develops a framework for data-driven analysis of work motions and studies their correlations to work speeds or execution rates, using data collected from modern motion sensors. The past analyses largely relied on manual steps involving time-consuming stop-watching and video-taping, followed by manual data analysis. While modern sensing devices have automated the collection of motion data, the motion analytics that transform the new data into knowledge are largely underdeveloped. Unsolved technical questions include: How the motion and time information can be extracted from the motion sensor data, how work motions and execution rates are statistically modeled and compared, and what are the statistical correlations of motions to the rates? In this paper, we develop a novel mathematical framework for motion and time analysis with motion sensor data, by defining new mathematical representation spaces of human motions and execution rates and by developing statistical tools on these new spaces. This methodological research is demonstrated using five use cases applied to manufacturing motion data., Keywords: motion and time study, motion sensors, Riemannian manifold, probability distribution on manifold, temporal evolution of probability distributions

Bayesian optimization is a powerful tool for expensive stochastic black-box optimization problems such as simulation-based optimization or machine learning hyperparameter tuning. Many stochastic objective functions implicitly require a random number seed as input. By explicitly reusing a seed a user can exploit common random numbers, comparing two or more inputs under the same randomly generated scenario, such as a common customer stream in a job shop problem, or the same random partition of training data into training and validation set for a machine learning algorithm. With the aim of finding an input with the best average performance over infinitely many seeds, we propose a novel Gaussian process model that jointly models both the output for each seed and the average. We then introduce the Knowledge gradient for Common Random Numbers that iteratively determines a combination of input and random seed to evaluate the objective and automatically trades off reusing old seeds and querying new seeds, thus overcoming the need to evaluate inputs in batches or measuring differences of pairs as suggested in previous methods. We investigate the Knowledge Gradient for Common Random Numbers both theoretically and empirically, finding it achieves significant performance improvements with only moderate added computational cost., Comment: 21 pages + Appendix, under review

Peak/off-peak spreads on European electricity forward and spot markets are eroding due to the ongoing nuclear phaseout in Germany and the steady growth in photovoltaic capacity. The reduced profitability of peak/off-peak arbitrage forces hydropower producers to recover part of their original profitability on the reserve markets. We propose a bi-layer stochastic programming framework for the optimal operation of a fleet of interconnected hydropower plants that sells energy on both the spot and the reserve markets. The outer layer (the planner's problem) optimizes end-of-day reservoir filling levels over one year, whereas the inner layer (the trader's problem) selects optimal hourly market bids within each day. Using an information restriction whereby the planner prescribes the end-of-day reservoir targets one day in advance, we prove that the trader's problem simplifies from an infinite-dimensional stochastic program with 25 stages to a finite two-stage stochastic program with only two scenarios. Substituting this reformulation back into the outer layer and approximating the reservoir targets by affine decision rules allows us to simplify the planner's problem from an infinite-dimensional stochastic program with 365 stages to a two-stage stochastic program that can conveniently be solved via the sample average approximation. Numerical experiments based on a cascade in the Salzburg region of Austria demonstrate the effectiveness of the suggested framework.

Platooning of vehicles is a promising approach for reducing fuel consumption, increasing vehicle safety, and using road space more efficiently. We consider the important but difficult problem of assigning optimal routes and departure schedules to a collection of vehicles. We propose an iterative route-then-schedule heuristic for centralized planning that quickly converges to high-quality solutions. We also propose and analyze a collection of valid inequalities for the individual problems of assigning vehicles to routes and scheduling the times that vehicles traverse their routes. These inequalities are shown to reduce the computational time or optimality gap of solving the routing and scheduling problem instances. Our approach uses the valid inequalities in both the routing and scheduling portions of each iteration; numerical experiments highlight the speed of the approach for routing vehicles on a real-world road network., 61 pages, 7 figures

This paper studies a setting in which multiple suppliers compete for a buyer's procurement business. The buyer faces uncertain demand and there is a requirement to reserve capacity in advance of knowing the demand. Each supplier has costs that are two dimensional, with some costs incurred before demand is realized in order to reserve capacity and some costs incurred after demand is realized at the time of delivery. A distinctive feature of our model is that the marginal costs may not be constants, and this naturally leads us to a supply function competition framework in which each supplier offers a schedule of prices and quantities. We treat this problem as an example of a general class of capacity games and show that, when the optimal supply chain profit is submodular, in equilibrium the buyer makes a reservation choice that maximizes the overall supply chain profit, each supplier makes a profit equal to their marginal contribution to the supply chain, and the buyer takes the remaining profit. We further prove that this submodularity property holds under two commonly studied settings: (1) there are only two suppliers; and (2) in the case of more than two suppliers, the marginal two-dimension costs of each supplier are non-decreasing and constant, respectively., Comment: 35 pages

We study a noisy tensor completion problem of broad practical interest, namely, the reconstruction of a low-rank tensor from highly incomplete and randomly corrupted observations of its entries. While a variety of prior work has been dedicated to this problem, prior algorithms either are computationally too expensive for large-scale applications, or come with sub-optimal statistical guarantees. Focusing on "incoherent" and well-conditioned tensors of a constant CP rank, we propose a two-stage nonconvex algorithm -- (vanilla) gradient descent following a rough initialization -- that achieves the best of both worlds. Specifically, the proposed nonconvex algorithm faithfully completes the tensor and retrieves all individual tensor factors within nearly linear time, while at the same time enjoying near-optimal statistical guarantees (i.e. minimal sample complexity and optimal estimation accuracy). The estimation errors are evenly spread out across all entries, thus achieving optimal $\ell_{\infty}$ statistical accuracy. We have also discussed how to extend our approach to accommodate asymmetric tensors. The insight conveyed through our analysis of nonconvex optimization might have implications for other tensor estimation problems., Comment: Accepted to Operations Research

In this paper we prove the efficacy of a simple greedy algorithm for a finite horizon online resource allocation/matching problem, when the corresponding static planning linear program (SPP) exhibits a non-degeneracy condition called the general position gap (GPG). The key intuition that we formalize is that the solution of the reward maximizing SPP is the same as a feasibility Linear Program restricted to the optimal basic activities, and under GPG this solution can be tracked with bounded regret by a greedy algorithm, i.e., without the commonly used technique of periodically resolving the SPP. The goal of the decision maker is to combine resources (from a finite set of resource types) into configurations (from a finite set of feasible configurations) where each configuration is specified by the number of resources consumed of each type and a reward. The resources are further subdivided into three types - offline (whose quantity is known and available at time 0), online-queueable (which arrive online and can be stored in a buffer), and online-nonqueueable (which arrive online and must be matched on arrival or lost). Under GRG we prove that, (i) our greedy algorithm gets bounded any-time regret of $\mathcal{O}(1/\epsilon_0)$ for matching reward ($\epsilon_0$ is a measure of the GPG) when no configuration contains both an online-queueable and an online-nonqueueable resource, and (ii) $\mathcal{O}(\log t)$ expected any-time regret otherwise (we also prove a matching lower bound). By considering the three types of resources, our matching framework encompasses several well-studied problems such as dynamic multi-sided matching, network revenue management, online stochastic packing, and multiclass queueing systems., Comment: 48 pages

Wasserstein distributionally robust optimization (DRO) has recently achieved empirical success for various applications in operations research and machine learning, owing partly to its regularization effect. Although connection between Wasserstein DRO and regularization has been established in several settings, existing results often require restrictive assumptions, such as smoothness or convexity, that are not satisfied for many problems. In this paper, we develop a general theory on the variation regularization effect of the Wasserstein DRO - a new form of regularization that generalizes total-variation regularization, Lipschitz regularization and gradient regularization. Our results cover possibly non-convex and non-smooth losses and losses on non-Euclidean spaces. Examples include multi-item newsvendor, portfolio selection, linear prediction, neural networks, manifold learning, and intensity estimation for Poisson processes, etc. As an application of our theory of variation regularization, we derive new generalization guarantees for adversarial robust learning., Comment: The paper is previously titled "Wasserstein Distributional Robustness and Regularization in Statistical Learning"

Multistage stochastic programs can be approximated by restricting policies to follow decision rules. Directly applying this idea to problems with integer decisions is difficult because of the need for decision rules that lead to integral decisions. In this work, we introduce Lagrangian dual decision rules (LDDRs) for multistage stochastic mixed-integer programming (MSMIP) which overcome this difficulty by applying decision rules in a Lagrangian dual of the MSMIP. We propose two new bounding techniques based on stagewise (SW) and nonanticipative (NA) Lagrangian duals where the Lagrangian multiplier policies are restricted by LDDRs. We demonstrate how the solutions from these duals can be used to drive primal policies. Our proposal requires fewer assumptions than most existing MSMIP methods. We compare the theoretical strength of the restricted duals and show that the restricted NA dual can provide relaxation bounds at least as good as the ones obtained by the restricted SW dual. In our numerical study on two problem classes, one traditional and one novel, we observe that the proposed LDDR approaches yield significant optimality gap reductions compared to existing general-purpose bounding methods for MSMIP problems.

In this paper, we describe the first mixed-integer nonlinear programming (MINLP) based solution approach that successfully identifies the most energy-efficient distillation configuration sequence for a given separation. Current sequence design strategies are largely heuristic. The rigorous approach presented here can help reduce the significant energy consumption and consequent greenhouse gas emissions by separation processes, where crude distillation alone is estimated to consume 6.9 quads of energy per year globally. The challenge in solving this problem arises from the large number of feasible configuration sequences and because the governing equations contain non-convex fractional terms. We make several advances to enable solution of these problems. First, we model discrete choices using a formulation that is provably tighter than previous formulations. Second, we highlight the use of partial fraction decomposition alongside Reformulation-Linearization Technique (RLT). Third, we obtain convex hull results for various special structures. Fourth, we develop new ways to discretize the MINLP. Finally, we provide computational evidence to demonstrate that our approach significantly outperforms the state-of-the-art techniques., 50 pages, 12 figures

We consider an auction market in which market makers fill the order book during a given time period while some other investors send market orders. We define the clearing price of the auction as the price maximizing the exchanged volume at the clearing time according to the supply and demand of each market participants. Then we derive in a semi-explicit form the error made between this clearing price and the fundamental price as a function of the auction duration. We study the impact of the behavior of market takers on this error. To do so we consider the case of naive market takers and that of rational market takers playing a Nash equilibrium to minimize their transaction costs. We compute the optimal duration of the auctions for 77 stocks traded on Euronext and compare the quality of price formation process under this optimal value to the case of a continuous limit order book. Continuous limit order books are found to be usually sub-optimal. However, in term of our metric, they only moderately impair the quality of price formation process. Order of magnitude of optimal auction durations is from 2 to 10 minutes.

We develop a framework for designing simple and efficient policies for a family of online allocation and pricing problems, that includes online packing, budget-constrained probing, dynamic pricing, and online contextual bandits with knapsacks. In each case, we evaluate the performance of our policies in terms of their regret (i.e., additive gap) relative to an offline controller that is endowed with more information than the online controller. Our framework is based on Bellman Inequalities, which decompose the loss of an algorithm into two distinct sources of error: (1) arising from computational tractability issues, and (2) arising from estimation/prediction of random trajectories. Balancing these errors guides the choice of benchmarks, and leads to policies that are both tractable and have strong performance guarantees. In particular, in all our examples, we demonstrate constant-regret policies that only require re-solving an LP in each period, followed by a simple greedy action-selection rule; thus, our policies are practical as well as provably near optimal., Comment: To appear in Operations Research, 2020

Piecewise linear functions are deceptively simple structures that are nonetheless capable of approximating complex nonlinear behavior. As such, they have been adopted throughout operations research and engineering to approximate nonlinear structures in optimization problems which would otherwise render the problem extremely difficult to solve. In “Nonconvex Piecewise Linear Functions: Advanced Formulations and Simple Modeling Tools,” J. Huchette and J. P. Vielma derive new mixed-integer programming (MIP) formulations for embedding low-dimensional nonconvex piecewise linear functions in optimization models. These formulations computationally outperform the crowded field of existing approaches in a number of regimes of interest. As these formulations are derived using recently developed machinery that produce highly performant, but uninterpretable, formulations, the authors showcase the utility of high-level modeling tools by presenting PiecewiseLinearOpt.jl, an extension to the popular JuMP optimization modeling language that implements a host of MIP formulations for piecewise linear function in a single, easy-to-use interface.

Tree ensemble models such as random forests and boosted trees are among the most widely used and practically successful predictive models in applied machine learning and business analytics. Although such models have been used to make predictions based on exogenous, uncontrollable independent variables, they are increasingly being used to make predictions where the independent variables are controllable and are also decision variables. In this paper, we study the problem of tree ensemble optimization: given a tree ensemble that predicts some dependent variable using controllable independent variables, how should we set these variables so as to maximize the predicted value? We formulate the problem as a mixed-integer optimization problem. We theoretically examine the strength of our formulation, provide a hierarchy of approximate formulations with bounds on approximation quality and exploit the structure of the problem to develop two large-scale solution methods, one based on Benders decomposition and one based on iteratively generating tree split constraints. We test our methodology on real data sets, including two case studies in drug design and customized pricing, and show that our methodology can efficiently solve large-scale instances to near or full optimality, and outperforms solutions obtained by heuristic approaches. In our drug design case, we show how our approach can identify compounds that efficiently trade-off predicted performance and novelty with respect to existing, known compounds. In our customized pricing case, we show how our approach can efficiently determine optimal store-level prices under a random forest model that delivers excellent predictive accuracy., 49 pages, 10 figures; to appear in Operations Research

The lack of emergency medical transportation is viewed as the main barrier to the access and availability of emergency medical care in low- and middle-income countries (LMICs). In this paper, we present a robust optimization approach to optimize both the location and routing of emergency response vehicles, accounting for uncertainty in travel times and spatial demand characteristic of LMICs. We traveled to Dhaka, Bangladesh, the sixth largest and third most densely populated city in the world, to conduct field research resulting in the collection of two unique data sets that inform our approach. These data are leveraged to estimate demand for emergency medical services in an LMIC setting and to predict the travel time between any two locations in the road network for different times of day and days of the week. We combine our prediction-optimization framework with a simulation model and real data to provide an in-depth investigation into three policy-related questions. First, we demonstrate that outpost locations optimized for weekday rush hour lead to good performance for all times of day and days of the week. Second, we find that the performance of the current system could be replicated using one third of the current outpost locations and one half of the current number of ambulances. Finally, we show that a fleet of small ambulances has the potential to significantly outperform traditional ambulance vans. In particular, they are able to capture approximately three times more demand while reducing the median average response time by roughly 10%–18% over the entire week and 24%–35% during rush hour because of increased routing flexibility offered by more nimble vehicles on a larger road network. Our results provide practical insights for emergency response optimization that can be leveraged by hospital-based and private ambulance providers in Dhaka and other urban centers in developing countries.

Studies on simulation input uncertainty are often built on the availability of input data. In this paper, we investigate an inverse problem where, given only the availability of output data, we nonparametrically calibrate the input models and other related performance measures of interest. We propose an optimization-based framework to compute statistically valid bounds on input quantities. The framework utilizes constraints that connect the statistical information of the real-world outputs with the input–output relation via a simulable map. We analyze the statistical guarantees of this approach from the view of data-driven distributionally robust optimization, and show how they relate to the function complexity of the constraints arising in our framework. We investigate an iterative procedure based on a stochastic quadratic penalty method to approximately solve the resulting optimization. We conduct numerical experiments to demonstrate our performances in bounding the input models and related quantities.

The clustering of a data set is one of the core tasks in data analytics. Many clustering algorithms exhibit a strong contrast between a favorable performance in practice and bad theoretical worst-cases. Prime examples are least-squares assignments and the popular $k$-means algorithm. We are interested in this contrast and study it through polyhedral theory. Several popular clustering algorithms can be connected to finding a vertex of the so-called bounded-shape partition polytopes. The vertices correspond to clusterings with extraordinary separation properties, in particular allowing the construction of a separating power diagram, defined by its so-called sites, such that each cluster has its own cell. First, we quantitatively measure the space of all sites that allow construction of a separating power diagram for a clustering by the volume of the normal cone at the corresponding vertex. This gives rise to a new quality criterion for clusterings, and explains why good clusterings are also the most likely to be found by some classical algorithms. Second, we characterize the edges of the bounded-shape partition polytopes. Through this, we obtain an explicit description of the normal cones. This allows us to compute measures with respect to the new quality criterion, and even compute "most stable" sites, and thereby "most stable" power diagrams, for the separation of clusters. The hardness of these computations depends on the number of edges incident to a vertex, which may be exponential. However, the computational effort is rewarded with a wealth of information that can be gained from the results, which we highlight through some proof-of-concept computations.

We introduce a framework for approximate dynamic programming that we apply to discrete time chains on $\mathbb{Z}_+^d$ with countable action sets. Our approach is grounded in the approximation of the (controlled) chain's generator by that of another Markov process. In simple terms, our approach stipulates applying a second-order Taylor expansion to the value function to replace the Bellman equation with one in continuous space and time where the transition matrix is reduced to its first and second moments. In some cases, the resulting equation (which we label {\bf TCP}) can be interpreted as corresponding to a Brownian control problem. When tractable, the TCP serves as a useful modeling tool. More generally, the TCP is a starting point for approximation algorithms. We develop bounds on the optimality gap---the sub-optimality introduced by using the control produced by the "Taylored" equation. These bounds can be viewed as a conceptual underpinning, analytical rather than relying on weak convergence arguments, for the good performance of controls derived from Brownian control problems. We prove that, under suitable conditions and for suitably "large" initial states, (i) the optimality gap is smaller than a $1-\alpha$ fraction of the optimal value, where $\alpha\in (0,1)$ is the discount factor, and (ii) the gap can be further expressed as the infinite horizon discounted value with a "lower-order" per period reward. Computationally, our framework leads to an "aggregation" approach with performance guarantees. While the guarantees are grounded in PDE theory, the practical use of this approach requires no knowledge of that theory.

In recent years, the paradigm of cloud computing has emerged as an architecture for computing that makes use of distributed (networked) computing resources. In this paper, we consider a distributed computing algorithmic scheme for stochastic optimization which relies on modest communication requirements amongst processors and most importantly, does not require synchronization. Specifically, we analyze a scheme with $N>1$ independent threads implementing each a stochastic gradient algorithm. The threads are coupled via a perturbation of the gradient (with attractive and repulsive forces) in a similar manner to mathematical models of flocking, swarming and other group formations found in nature with mild communication requirements. When the objective function is convex, we show that a flocking-like approach for distributed stochastic optimization provides a noise reduction effect similar to that of a centralized stochastic gradient algorithm based upon the average of $N$ gradient samples at each step. The distributed nature of flocking makes it an appealing computational alternative. We show that when the overhead related to the time needed to gather $N$ samples and synchronization is not negligible, the flocking implementation outperforms a centralized stochastic gradient algorithm based upon the average of $N$ gradient samples at each step. When the objective function is not convex, the flocking-based approach seems better suited to escape locally optimal solutions due to the repulsive force which enforces a certain level of diversity in the set of candidate solutions. Here again, we show that the noise reduction effect is similar to that associated to the centralized stochastic gradient algorithm based upon the average of $N$ gradient samples at each step., To appear in Operations Research. Copyright: 2017 INFORMS

The goal of ranking and selection (R&S) procedures is to identify the best stochastic system from among a finite set of competing alternatives. Such procedures require constructing estimates of each system’s performance, which can be obtained simultaneously by running multiple independent replications on a parallel computing platform. Nontrivial statistical and implementation issues arise when designing R&S procedures for a parallel computing environment. We propose several design principles for parallel R&S procedures that preserve statistical validity and maximize core utilization, especially when large numbers of alternatives or cores are involved. These principles are followed closely by our parallel Good Selection Procedure (GSP), which, under the assumption of normally distributed output, (i) guarantees to select a system in the indifference zone with high probability, (ii) in tests on up to 1,024 parallel cores runs efficiently, and (iii) in an example uses smaller sample sizes compared to existing parallel procedures, particularly for large problems (over 106 alternatives). In our computational study we discuss three methods for implementing GSP on parallel computers, namely the Message-Passing Interface (MPI), Hadoop MapReduce, and Spark, and show that Spark provides a good compromise between the efficiency of MPI and robustness to core failures. The e-companion is available at https://doi.org/10.1287/opre.2016.1577 .

Many sequential decision problems can be formulated as Markov Decision Processes (MDPs) where the optimal value function (or cost-to-go function) can be shown to satisfy a monotone structure in some or all of its dimensions. When the state space becomes large, traditional techniques, such as the backward dynamic programming algorithm (i.e., backward induction or value iteration), may no longer be effective in finding a solution within a reasonable time frame, and thus we are forced to consider other approaches, such as approximate dynamic programming (ADP). We propose a provably convergent ADP algorithm called Monotone-ADP that exploits the monotonicity of the value functions in order to increase the rate of convergence. In this paper, we describe a general finite-horizon problem setting where the optimal value function is monotone, present a convergence proof for Monotone-ADP under various technical assumptions, and show numerical results for three application domains: optimal stopping, energy storage/allocation, and glycemic control for diabetes patients. The empirical results indicate that by taking advantage of monotonicity, we can attain high quality solutions within a relatively small number of iterations, using up to two orders of magnitude less computation than is needed to compute the optimal solution exactly., Comment: 35 pages, 11 figures

We consider a service system model primarily motivated by the problem of efficient assignment of virtual machines to physical host machines in a network cloud, so that the number of occupied hosts is minimized. There are multiple input flows of different type customers, with a customer mean service time depending on its type. There is infinite number of servers. A server packing {\em configuration} is the vector $k=\{k_i\}$, where $k_i$ is the number of type $i$ customers the server "contains". Packing constraints must be observed, namely there is a fixed finite set of configurations $k$ that are allowed. Service times of different customers are independent; after a service completion, each customer leaves its server and the system. Each new arriving customer is placed for service immediately; it can be placed into a server already serving other customers (as long as packing constraints are not violated), or into an idle server. We consider a simple parsimonious real-time algorithm, called {\em Greedy}, which attempts to minimize the increment of the objective function $\sum_k X_k^{1+\alpha}$, $\alpha>0$, caused by each new assignment; here $X_k$ is the number of servers in configuration $k$. (When $\alpha$ is small, $\sum_k X_k^{1+\alpha}$ approximates the total number $\sum_k X_k$ of occupied servers.) Our main results show that certain versions of the Greedy algorithm are {\em asymptotically optimal}, in the sense of minimizing $\sum_k X_k^{1+\alpha}$ in stationary regime, as the input flow rates grow to infinity. We also show that in the special case when the set of allowed configurations is determined by {\em vector-packing} constraints, Greedy algorithm can work with {\em aggregate configurations} as opposed to exact configurations $k$, thus reducing computational complexity while preserving the asymptotic optimality., Comment: 22 pages

Even in the face of deteriorating and highly volatile demand, firms often invest in, rather than discard, aging technologies. To study this phenomenon, we model the firm's profit stream as a Brownian motion with negative drift. At each point in time, the firm can continue operations, or it can stop and exit the project. In addition, there is a one-time option to make an investment that boosts the project's profit rate. Using stochastic analysis, we show that the optimal policy always exists and that it is characterized by three thresholds. There are investment and exit thresholds before investment, and there is a threshold for exit after investment. We also effect a comparative statics analysis of the thresholds with respect to the drift and the volatility of the Brownian motion. When the profit boost upon investment is sufficiently large, we find a novel result: the investment threshold decreases in volatility.

We study the general approach to accelerating the convergence of the most widely used solution method of Markov decision processes with the total expected discounted reward. Inspired by the monotone behavior of the contraction mappings in the feasible set of the linear programming problem equivalent to the MDP, we establish a class of operators that can be used in combination with a contraction mapping operator in the standard value iteration algorithm and its variants. We then propose two such operators, which can be easily implemented as part of the value iteration algorithm and its variants. Numerical studies show that the computational savings can be significant especially when the discount factor approaches 1 and the transition probability matrix becomes dense, in which the standard value iteration algorithm and its variants suffer from slow convergence., 32 pages, 2 figures, 2 table