Search

Data centers are significant contributors to carbon emissions and can strain power systems due to their high electricity consumption. To mitigate this impact and to participate in demand response programs, cloud computing companies strive to balance and optimize operations across their global fleets by making strategic decisions about when and where to place compute jobs for execution. In this paper, we introduce a load shaping scheme which reacts to time-varying grid signals by leveraging both temporal and spatial flexibility of compute jobs to provide risk-aware management guidelines and job placement with provable performance guarantees based on distributionally robust optimization. Our approach divides the problem into two key components: (i) day-ahead planning, which generates an optimal scheduling strategy based on historical load data, and (ii) real-time job placement and (time) scheduling, which dynamically tracks the optimal strategy generated in (i). We validate our method in simulation using normalized load profiles from randomly selected Google clusters, incorporating time-varying grid signals. We can demonstrate significant reductions in carbon cost and peak power with our approach compared to myopic greedy policies, while maintaining computational efficiency and abiding to system constraints.

The amount of CO$_2$ emitted per kilowatt-hour on an electricity grid varies by time of day and substantially varies by location due to the types of generation. Networked collections of warehouse scale computers, sometimes called Hyperscale Computing, emit more carbon than needed if operated without regard to these variations in carbon intensity. This paper introduces Google's system for Carbon-Intelligent Compute Management, which actively minimizes electricity-based carbon footprint and power infrastructure costs by delaying temporally flexible workloads. The core component of the system is a suite of analytical pipelines used to gather the next day's carbon intensity forecasts, train day-ahead demand prediction models, and use risk-aware optimization to generate the next day's carbon-aware Virtual Capacity Curves (VCCs) for all datacenter clusters across Google's fleet. VCCs impose hourly limits on resources available to temporally flexible workloads while preserving overall daily capacity, enabling all such workloads to complete within a day. Data from operation shows that VCCs effectively limit hourly capacity when the grid's energy supply mix is carbon intensive and delay the execution of temporally flexible workloads to "greener" times.

Datacenter power demand has been continuously growing and is the key driver of its cost. An accurate mapping of compute resources (CPU, RAM, etc.) and hardware types (servers, accelerators, etc.) to power consumption has emerged as a critical requirement for major Web and cloud service providers. With the global growth in datacenter capacity and associated power consumption, such models are essential for important decisions around datacenter design and operation. In this paper, we discuss two classes of statistical power models designed and validated to be accurate, simple, interpretable and applicable to all hardware configurations and workloads across hyperscale datacenters of Google fleet. To the best of our knowledge, this is the largest scale power modeling study of this kind, in both the scope of diverse datacenter planning and real-time management use cases, as well as the variety of hardware configurations and workload types used for modeling and validation. We demonstrate that the proposed statistical modeling techniques, while simple and scalable, predict power with less than 5% Mean Absolute Percent Error (MAPE) for more than 95% diverse Power Distribution Units (more than 2000) using only 4 features. This performance matches the reported accuracy of the previous started-of-the-art methods, while using significantly less features and covering a wider range of use cases.

The paper presents accurate and simple dynamic model of a supercapacitor bank system for power system dynamics studies. The proposed model is derived from a detailed RC circuit representation. Furthermore, a complete control system of the supercapacitor bank is also presented. The proposed model is easy to integrate in any power system simulation software and consists of only up to four standard datasheet parameters. The performance of the proposed model in grid frequency control and low-voltage ride through is illustrated on IEEE 14-bus test system in DIgSILENT PowerFactory. It is shown that in case of transient stability simulations the ideal (simplified) model of the supercapacitor can be used while in case of frequency control the ideal representation may not always be appropriate.

Electricity market price predictions enable energy market participants to shape their consumption or supply while meeting their economic and environmental objectives. By utilizing the basic properties of the supply-demand matching process performed by grid operators, known as Optimal Power Flow (OPF), we develop a methodology to recover energy market's structure and predict the resulting nodal prices by using only publicly available data, specifically grid-wide generation type mix, system load, and historical prices. Our methodology uses the latest advancements in statistical learning to cope with high dimensional and sparse real power grid topologies, as well as scarce, public market data, while exploiting structural characteristics of the underlying OPF mechanism. Rigorous validations using the Southwest Power Pool (SPP) market data reveal a strong correlation between the grid level mix and corresponding market prices, resulting in accurate day-ahead predictions of real time prices. The proposed approach demonstrates remarkable proximity to the state-of-the-art industry benchmark while assuming a fully decentralized, market-participant perspective. Finally, we recognize the limitations of the proposed and other evaluated methodologies in predicting large price spike values., Comment: 14 pages, 14 figures. Accepted for publication in IEEE Transactions on Power Systems

Bin packing is an algorithmic problem that arises in diverse applications such as remnant inventory systems, shipping logistics, and appointment scheduling. In its simplest variant, a sequence of $T$ items (e.g., orders for raw material, packages for delivery) is revealed one at a time, and each item must be packed on arrival in an available bin (e.g., remnant pieces of raw material in inventory, shipping containers). The sizes of items are i.i.d. samples from an unknown distribution, but the sizes are known when the items arrive. The goal is to minimize the number of non-empty bins (equivalently waste, defined to be the total unused space in non-empty bins). This problem has been extensively studied in the Operations Research and Theoretical Computer Science communities, yet all existing heuristics either rely on learning the distribution or exhibit $o(T)$ additive suboptimality compared to the optimal offline algorithm only for certain classes of distributions (those with sublinear optimal expected waste). In this paper, we propose a family of algorithms which are the first truly distribution-oblivious algorithms for stochastic bin packing, and achieve $\mathcal{O}(\sqrt{T})$ additive suboptimality for all item size distributions. Our algorithms are inspired by approximate interior-point algorithms for convex optimization. In addition to regret guarantees for discrete i.i.d. sequences, we extend our results to continuous item size distribution with bounded density, and also prove a family of novel regret bounds for non-i.i.d. input sequences. To the best of our knowledge these are the first such results for non-i.i.d. and non-random-permutation input sequences for online stochastic packing.

It is well known that the static caching algorithm that keeps the most frequently requested documents in the cache is optimal in case when documents are of the same size and requests are independent and equally distributed. However, it is hard to develop explicit and provably optimal caching algorithms when requests are statistically correlated. In this paper, we show that keeping the most frequently requested documents in the cache is still optimal for large cache sizes even if the requests are strongly correlated.

The analysis of stochastic loss networks has long been of interest in computer and communications networks and is becoming important in the areas of service and information systems. In traditional settings, computing the well known Erlang formula for blocking probability in these systems becomes intractable for larger resource capacities. Using compound point processes to capture stochastic variability in the request process, we generalize existing models in this framework and derive simple asymptotic expressions for blocking probabilities. In addition, we extend our model to incorporate reserving resources in advance. Although asymptotic, our experiments show an excellent match between derived formulas and simulation results even for relatively small resource capacities and relatively large values of blocking probabilities., Comment: This is the longer version of the same titled paper that will appear on Journal of Applied Probability