Search

The power grid is undergoing a transformative shift towards renewable and distributed energy resources (DERs) connected through power electronic inverters. This transition plays a vital role in addressing the climate crisis, reducing greenhouse gas emissions, and enhancing energy supply reliability. While these advancements improve grid resilience through localized generation, they also introduce challenges related to variability and intermittency, especially in frequency stability. Inverter-based DERs provide rapid response capabilities essential for functions like frequency regulation, requiring robust control strategies, advanced communication systems, and sophisticated algorithms for optimal resource allocation and real-time monitoring.This research leverages inverter-based DERs for grid services, emphasizing real-world testing and validation. Chapter 1 integrates second-life electric vehicle (EV) batteries into grid service provision, addressing battery disposal and resource waste. We develop a control framework for a heterogeneous battery unifying system that optimizes the state of health of second-life batteries while delivering energy services. Our optimization algorithm minimizes reconditioning time and ensures reliability, validated through hardware-in-loop testing with Nissan Leaf batteries.The research further extends to system-level frequency control strategies across distribution and transmission grids. In Chapter 2, we conduct real-world testing with 69 active and 107 passive DERs on the UCSD microgrid, overcoming technical barriers and evaluating both cyber and physical performance for distributed secondary frequency control. In Chapter 3, we introduce a decentralized method for dynamic inertia allocation that enhances real-time frequency stability in low-inertia transmission grids, utilizing instantaneous local frequency measurements and available inertia estimates to stabilize frequency within milliseconds.Finally, Chapter 4 validates an optimal frequency controller for low and variable inertia transmission grids through real-time and electromagnetic transient-based methods. Key contributions include improved simulation accuracy and the effective use of microgrid grid-forming inverters for disturbance mitigation, providing precise real-time performance guarantees.By connecting theoretical insights with practical applications, this research enhances the resilience and efficiency of the power grid, ultimately supporting a sustainable energy future and contributing to global efforts to mitigate climate change.

The push for multi-sector decarbonization in the electric grids around the world has created the need for significant investment in the electrical grid. Over the next two decades, the bulk power system will be transformed from being historically dominated by gas-fired generation to dominated by wind, solar, and energy storage resources. Due to the large costs associated with investment and operation, and the importance of maintaining reliability in this critical infrastructure, a comprehensive investment plan is crucial. This dissertation looks at the California power system decarbonization problem through several lenses. The first concerns a novel formulation for decarbonization planning which incorporates more detailed modeling of gas-fired generation, and an advanced solution algorithm which enables its practical utility. Then, several novel methods for temporal sampling in these models are proposed. Finally, two case studies are considered. The first examines the impact of smart charging for medium- and heavy-duty battery electric vehicles on the decarbonization plan. The second concerns scenario analysis around climate change pathways. At the same time, a novel method for ensuring reliable power system investments, with specific regard to decarbonized portfolios, is proposed.

This dissertation focuses on devising innovative approaches to understanding and computing the optimal decision within complex large-scale operational frameworks, such as service networks and power systems. These approaches aim to facilitate interpretable, scalable, and robust decision-making under uncertainty. However, achieving this objective is hindered by the computational hurdle presented by the fact that most practical problems are known to be NP-hard in the worst case. Nevertheless, this dissertation has been primarily inspired by the observation that real-world problem instances often exhibit benign geometric properties, which can be exploited to reduce the computational complexity. By leveraging methodologies from mathematics, statistics, operations research, and machine learning, our goal is to identify and harness these underlying geometric properties to devise algorithms that are demonstrably efficient and resilient, thereby supporting decision-making in large-scale systems.In the first part of the dissertation, we focus on the low-rank matrix optimization problem, which targets at recovering the underlying low-rank ground truth matrix from a small number of measurements. Various important applications in the fields of machine learning, signal processing, and power systems can be formulated as a low-rank matrix optimization problem. By utilizing the benign optimization landscape around the manifold of low-rank matrices, we establish state-of-the-art theoretical guarantees to the highly efficient non-convex formulation based on the Burer-Monteiro factorization. First, we significantly improve existing sufficient conditions in terms of the Restricted Isometry Property constant, leading to the guaranteed success of local search algorithms for more problem instances. Next, we propose a new complexity metric for the rank-1 generalized matrix completion problem. The new metric has the potential of unifying several existing metrics and provides both sufficient conditions and necessary conditions to the success of local search algorithms. This part serves as a crucial step towards closing the gap between the empirical success and the theoretical understanding of the Burer-Monteiro factorization approach. The second part of the dissertation is concerned with the discrete optimization via simulation problem. The design of scalable and robust simulation-optimization algorithms plays a vital role in the timely decision-making in large-scale systems with uncertainty, such as the bike-sharing system. Inspired by the marginal decreasing property, we utilize a special structure, named the L-convexity, to develop algorithms with scalability and optimality guarantees. More specifically, we first construct a subgradient estimator based on the Lovasz extension and develop stochastic search algorithms using the subgradient information. We theoretically and empirically illustrate that the proposed algorithms are highly efficient for high-dimensional discrete optimization via simulation problems. Next, by combining the subgradient information with the discrete nature of the problem, we propose the stochastic localization algorithms, which exhibit an improved efficiency on large-scale applications.In the third part of the dissertation, we focus on the AC power flow problem in power systems. The efficient and reliable control of large-scale power systems, e.g., the dispatch of electricity under safety-critical constraints, is contingent upon developments of advanced computational and analytical tools. In the first chapter, we consider the uniqueness of solutions to the power flow problem. Utilizing properties of the monotone regime and the network topology, stronger necessary conditions and sufficient conditions, based on the maximal girth and the maximal eye, are proposed to guarantee the uniqueness of solutions. We also provide the corresponding reduction algorithms to efficiently estimate these two network parameters. Given the ongoing emergence of intermittent renewable generation, the second chapter is devoted to the design of optimal power flow algorithms that are robust to large generation forecast errors, which plays an essential role in incorporating renewable energy generators into electrical networks. More concretely, we propose a novel distributionally robust optimization formulation for the chance-constrained optimal power flow problem and provide corresponding algorithms to effectively find the robust solution with the minimum generation cost.

Customers are adopting new technologies, fundamentally changing their relationship with the electrical grid. These technologies, known as distributed energy resources (DERs), allow customers to control their consumption. They give customers the flexibility to purchase energy when prices are lower and sometimes sell electricity back to the grid when prices are high. While DERs can benefit the individual customer and potentially contribute to decarbonization, they also require electrical utility companies to change how they operate the grid, especially in the distribution system. This dissertation examines the relationship between DER operations and the electrical distribution system. We first explore how “business as usual” approaches to DERs could negatively impact the grid. Specifically, how current electricity tariffs provide an incentive structure for customers to optimize their DERs and how their optimized consumption could lead to congestion issues on the distribution system. Then, we explore data-driven tools for addressing this congestion. We collect data from simulated smart meters and develop a voltage estimator to manage congestion through a centralized optimized DER dispatch approach. Finally, we propose an electricity tariff design that provides the benefits of centralized congestion management, but instead through price signals sent to customers.In Chapter 1, we explore the relationship between current electricity tariffs and future DER operations. We model commercial customers optimizing their electric vehicle (EV) charging under different Pacific Gas and Electric (PG&E) tariffs, including a new tariff (BEV tariff) with a novel subscription-based power charge. We model customers optimizing their load across different seasons to capture the prices in each tariff and compare the total annual costs. Once we calculate the optimal charging strategy for each customer, we model them in an open-source grid simulation software using a feeder model based on a PG\&E distribution feeder, measuring voltage and wholesale energy costs. We find that undervoltage would occur in our distribution feeder under all four tariffs we examine. This result indicates that current tariff designs are insufficient for congestion management, and additional measures are required. We also find that customers could see nearly a 15% reduction in costs by switching from one tariff to another. This cost savings for the customer is also reflected in a potentially problematic revenue gap for the utility when customers change tariffs. Finally, we show that subscription-based tariffs are less efficient than traditional demand charge-based tariffs, but the lower prices in the BEV tariff mask its inefficiency.In Chapter 2, we explore ways to add a voltage constraint to the same EV charging coordination problem as Chapter 1 for distribution system operators (DSOs) to perform direct, centralized control of DERs. This chapter presents a novel three-phase data-driven linear voltage magnitude estimator based on past smart meter and substation data. This estimator is trained offline solely on data readily available for the distribution system operator and reduces the size of the voltage constraint by only estimating voltage at customer service connections. We show that our voltage estimator can prevent undervoltage in the EV charging coordination problem and is faster and more accurate than three-phase linearized voltage approximations, requiring less computational memory and no knowledge of the distribution system network connectivity. In Chapter 3, we examine price-signal control methods for congestion management of DERs. We describe a new pricing scheme for day-ahead congestion management of DERs, referred to as Load Responsive Prices (LRPs). Using the LRP method, we can determine the optimal load profile of customers from offline analyses and then use day-ahead prices (potentially taken from the wholesale market) to calculate prices that customers can independently optimize for without additional communication from the DSO beyond the prices themselves. We find that customers optimizing using LRPs consume the same way as customers following direct load control methods but without knowing the load levels the system operator requires. We end the chapter with a discussion of various implementation considerations. We discuss ways to reduce customer cost increases, methods for addressing incorrect forecasts, and other important issues for practitioners.We conclude with a summary of findings, policy recommendations, and future research directions.

The increased electrification of society and calls for improvements in electrical grid reliability lend renewed attention to the domain of power systems optimization. For the future smart grid, it will be crucial both to find better solutions to classical power grid optimization problems such as optimal power flow (OPF) and state estimation (SE) as well as to develop new optimization methods for the updated physical systems. Due to the nonlinearity of alternating current (AC), many optimization problems in power systems are nonlinear and nonconvex and thus are hard to solve with existing fast convex methods. In this dissertation, we propose some efficient methods related to the OPF, post-contingency OPF, SE, and power flow (PF) problems. These methods provide new optimization and machine learning frameworks for ensuring the reliable and efficient operation of the power grid.

Electric power systems are one of the fundamental pillars of modern society. Power systems require careful planning to ensure enough capacity for future electric demand, and simultaneously requiring meticulous operation to maintain a continuous supply-demand balance, which ensures a reliable and stable system. In this complex context, our electric systems are being decarbonized, moving away from fossil fuel based systems into more renewable ones, with larger contribution from wind and solar energy sources, reducing emissions of greenhouse gases which negatively contribute to climate change. The consequences of this transition to renewable systems are multi-fold. First, renewable sources, like wind and solar, are variable and face uncertainty that complicates future planning of these energy systems. Second, the integration of Distributed Energy Resources (DERs), like rooftop photovoltaic systems, changes the paradigm of how our electric system operates, where power was once only generated in large generating units, and delivered via high-voltage transmission lines to the demand hubs—commonly large cities. Traditional energy sources, such as coal or gas, inject power via synchronous generators, however renewable resources are mostly interfaced into the grid using power electronics, that creates a fundamental difference in how electricity is produced. All of these changes are affecting our understanding of the grid, in multiple aspects and with different problems, which force us to reevaluate the tools and techniques used to study power systems.This work focus on understanding the different challenges that are occurring in power systems due to the integration of variable Renewable Energy Sources (RESs), both in planning studies and day-to-day system operation. Thus, the main goal of this thesis is to study and analyze the changes in power systems with increasing shares of RESs, across multiple time- and space- scales.The first part of this dissertation investigates dynamic simulations that are necessary for analyzing power system stability and dynamic response in the presence of Inverter-based Resources (IBRs). We introduce the Julia package PowerSimulationDynamics.jl to study the effects of load and line modeling when grid-forming and grid-following inverters. A discussion is presented between phasor-type and electromagnetic transient-type simulations. Results confirm that rooted assumptions in transient simulations may not be valid in systems with large presence of IBRs. By enabling PowerSimulationDynamics.jl as a flexible software tool, we discover that more detailed network and load models are becoming more necessary to properly assess stability of future power systems, as the integration of IBRs increases.The second part focuses on DER investment in peer-to-peer and sharing economy setups. In particular, we propose an optimization model for distributed rooftop Photovoltaic (PV) investment to analyze how PV investment decisions can vary when consumers are subjected to different tariff schemes. Our results showcase how peer-to-peer tariff schemes, rather than traditional net-metering or feed-in-tariffs, can promote investment in rooftop PV.Finally, the third part of this dissertation discuss the Switch expansion planning model, and specifically the effects of Electric Vehicles (EVs) flexibility in the future Western Electricity Coordinating Council (WECC) grid. Motivated by future decarbonization scenarios in California, we added a new EV module to the Switch model to study the influence of EV flexibility in installed capacity in WECC by 2050. Our results confirms that demand flexibility can reduce system's peak load to defer large investment decisions, achieving savings in both planning and operation costs.

Decision-makers often face environments which vary over time and deal with uncertainty as a result. Problems with temporal variation differ based on how frequently they are solved and what information about the present or future is available to the decision-maker; each class of problem poses distinct challenges and requires tailored solutions. In this dissertation, we design and analyze methods to support decision-makers in these settings, with an emphasis on applications in electric power systems.Online optimization, in which a player makes a sequence of decisions aimed at minimizing the damage inflicted by their adversary, is the first class of problems considered. Here, we shed light on online nonconvex optimization problems in which algorithms are evaluated against the optimal decision at each time using the notion of dynamic regret. The adversary's loss functions are arbitrarily nonconvex but have global solutions that are slowly time-varying. To address this problem we first analyze the region around the global solutions to define time-varying target sets. Then, we introduce two algorithms and prove that the dynamic regret for each algorithm is bounded by a function of the temporal variation in the optimal decision. The first algorithm assumes the decision-maker has some prior knowledge about the initial loss function. The second algorithm makes no assumption about prior knowledge, and instead it relies on random sampling and memory to find and then track the target sets over time. In this case, the landscape of the loss functions determines the likelihood that the dynamic regret will be small. Numerical experiments validate these theoretical results and highlight the impact of a single low-complexity problem early in the sequence.Time-varying optimization, in which a series of linked problems are solved sequentially using information about past decisions, but not considering the future, is the second class of problems considered. Specifically, we analyze the optimality behavior of solution trajectories for optimal power flow (OPF) with time-varying load. Despite its nonlinearity, time-varying OPF is commonly solved every 5-15 minutes using local-search algorithms. Failing to obtain the globally optimal solution of power optimization problems jeopardizes the grid’s reliability and causes poor financial and environmental outcomes. An empirical study on California data shows that, with enough variation in the data, local search methods can solve OPF to global optimality, even if the problem has many local minima. To explain this phenomenon, we introduce a backward mapping that relates the time-varying OPF’s global solution at a given time to a set of desirable initial points. We show that this mapping could act as a stochastic gradient ascent algorithm on an implicitly convexified formulation of OPF, justifying the escape of poor solutions over time. This work is the first to mathematically explain how temporal data variation affects the complexity of solving power operational problems. Multi-period optimization under uncertainty, in which decisions for the entire time horizon are made at once, is the third and final class of problems considered. Within this class, we focus on the decision-making problem facing hybrid power plant operators when participating in wholesale electricity markets with the goal of enabling researchers to accurately incorporate these resources into simulations of future electricity systems. To this end, a stochastic optimization model is developed to maximize the revenue of the plant, which consists of a renewable generator and an energy storage resource that appear as a single, combined unit to the market, while limiting risk due to uncertainty in market prices and generation levels, through price-quantity bid curves. The uncertainty is represented by scenarios, and a detailed methodology is provided for creating scenarios which both reflect the type of information a hybrid operator would have and only require the limited data which is available in simulation settings. This work advances existing models by allowing greater cooperation between the generator and storage components while also enforcing market limits on bid curves' complexity. Further, the approach ensures economically-sound behavior even when faced with unforeseen events.

Traditionally, utility distribution companies have treated electricity distribution networks as passive loads, assuming that the possibility of network constraint violations cannot be managed by controlling distributed energy resources (DERs) such as electric vehicle charging stations. The emergence of edge computing has made it possible for DER management systems (DERMS) to control the power injected/extracted by DERs in real time. By actively managing DERs when necessary, DERMS both increase interconnection availability and reduce the need for infrastructure upgrades. This dissertation presents several tools that can either make power-injection decisions for a DERMS or that support the system’s decision-making. These tools include Unsupervised Impedance and Topology Estimation, Multiple Model Adaptive Power System State Estimation, Voltage Phasor Control, Linearized Output Projected Gradient Descent Feedback Optimization, and Nullspace-Based Power Flow Linearization. The focus of the work is to develop theory which can explain when and how to use these tools on real distribution networks. A large portion of this dissertation develops the theory behind Nullspace-Based Power Flow Linearization, a novel interpretation of power flow linearization that explicitly considers constant-voltage buses, phasor angle-reference ambiguity, and power balance. This dissertation also includes a comment that was submitted in August 2022 on CPUC Proceeding R2207005. With Proceeding R2207005, the California Public Utilities Commission is investigating new rules and price structures for utilities which will change the future of DER management in California.

Climate change requires a radical and complex transition in the way our energy sector generates and uses energy. Solar and wind energy are leading carbon-free power sources, but they are non-dispatchable, meaning that an operator cannot control when their generation occurs. As a result, their growth may be stymied unless supporting technologies can change how the rest of our energy system responds to their generation. Demand response is an important tool in a suite of supporting technologies that help smooth the introduction of non-dispatchable energy into the grid. The more effective demand response signals (prices) are on responses (deferring of energy by appliances and building systems), the more quickly solar and wind may decarbonize our energy system. Demand response signals may be amplified by advanced controls, of which reinforcement learning is a prime example. We present two different environments for testing demand response price-setting at different levels of the grid. The first environment is MicrogridLearn, an environment that transmits prices to a collection of buildings and learns to set prices in a way that better shapes the energy and lowers energy costs for all buildings in its purview. The second environment is OfficeLearn, an environment that simulates behavioral response to prices. Using these two environments, we identify six key challenges in moving RL from simulation to reality, and present RL strategies, some novel, to overcome them. First, RL controllers implemented in the real world need to be data efficient: we present Offline-Online RL, Surprise-Minimizing RL, and extrinsic and intrinsic planning as solutions. Second, RL in the real world should be robust and guarantee safe action: we present a novel method, the \textit{guardrails} planning model, and demonstrate that using a conservative decision process with a distributional prediction can help learning. Third, RL may get stuck in local optima: we present meta-learning over domain randomization as a technique to ensure agent robustness. Fourth, agents may be attacked: we demonstrate a novel adversarial attack on RL and present a defense. Fifth, energy applications may require real-world RL to protect privacy and generalize to new subdomains easily; we present the first ever application of Personal Federated Hypernetworks (PFH) to RL to accomplish both tasks. Finally, hyperparameter sweeps may entail large data consumption; we present a regression analysis of hyperparameter sweep values to give a sense of hyperparameter-parameter strength. Finally, we discuss how RL may be implemented in experiment. We give a prospective experiment plan. We present an API to connect the RL controller to the real world. We then discuss two prior experiments run in the same office setting: an A/B test of two different energy visualizations, and a measure of the persistence of the effects of energy reduction after the experimental period ended. Our work contributes to societal knowledge in the following ways. We are the first to propose the use of RL for price-setting in energy systems, and we are the first to propose a Social Game as a mechanism to incentivize price sensitivity in an office setting. Of our RL methods, we are the first to propose adversarial poisoning during train time of algorithms. We are also the first to propose the use of personal federated hypernetworks for training new RL agents. Our other methods have been inspired by similar implementations in other fields, but are novel to the communities and application spaces in which we operate. We hope that, from our work, the community may continue to iterate on RL architectures for price-setting, and may implement these techniques in experiment.

To date, electric power systems are the most important technological achievement in the energy sector. Interconnected power systems require a careful operation to maintain supply stability and maintain efficient energy usage. Amid this complexity, two critical global transitions are needed: (1) moving away from greenhouse gases which negatively contribute to climate change; and (2) moving away from fossil fuels, which currently support transportation and heating systems. Computer simulations are the only tools available to do the necessary research and development required to accomplish these goals. However, the practices and tools used by the electric power industry were developed for operational situations that are vastly different from current challenges for energy transition. Variable Renewable Energy uncertainty, fast varying resources, and fundamental changes to the energy transformation physics require an update to the operational and simulation practices.This work started focusing on the simulation of future systems, and by the time it was written, it had become a dissertation on the practices needed for today's system. We discuss the requirements needed to improve scientific computing practices and conduct simulations more systematically when developing new operation models. The work has designed and implemented three software tools to develop simulations: the first includes PowerSystems.jl –– a software package that handles the data ingestion and processing required for simulations across multiple time scales. The second tool includes PowerSimulations.jl which resolves issues of model-limited choice when implementing operations simulations of large interconnected systems. The third tool includes PowerSimulations.jl, which is based on a formalization of the simulation of power system operations and provides a scalable computational platform. Third, PowerSimulationDynamics.jl which concentrates on the modeling of power systems dynamics with a strong focus on integrating Inverter Base Resources. PowerSystems.jl and PowerSimulations.jl both enable the development of a novel Automatic Generation Control model that can assess reserve deployments. Both tools are used to develop a Markovian Graph approach when integrating probabilistic forecasts into operators' risk assessments. Finally, this dissertation investigates the simulation techniques that are needed for system dynamics and challenges the applicability long-held simulation practices. It seeks to uncover the theoretical elements of simulation needs for the future grid. Results from PowerSimulationDynamics.jl confirm this dissertation's hypotheses and demonstrates the effectiveness of the simulation approach in replicating existing models. Furthermore, the findings in this thesis critically showcase the capabilities to simulate dynamics systems accurately using model transformations that reduces computational complexity without loss of accuracy.

As the momentum of climate action continues to shift from top-down, nation-state action to decentralized action by corporations, cities, and utilities, these actors are working to decarbonize their energy consumption and help drive the transition to a carbon-free grid. However, the current frameworks and metrics that drive voluntary decision making—GHG inventories and renewable energy procurement goals—do not always reflect the reality of the evolving power system, and thus can lead to decisions that do not effectively reduce emissions or address grid needs. This research seeks to fill the gaps in knowledge about how both regulatory and voluntary approaches to decarbonizing the electric grid can maximize their effectiveness and accurately measure and attribute emissions from electricity consumption. This dissertation draws upon power system engineering and industrial ecology research and applies data science and optimization methods to 1) identify whether there is a need to account for grid carbon emissions on an hourly basis, 2) introduce a comprehensive dataset of validated hourly emissions from the U.S. power sector, and 3) introduce a new modeling tool that enables greater understanding of the role of different voluntary clean energy procurement goals in the broader energy transition.

State estimation is an important tool for continuously monitoring the power system and aims to recover the underlying system voltage phasors, given supervisory control and data acquisition (SCADA) measurements and a model that encompasses the system topology and specifications. To ensure an accurate state estimation, it is essential to have the capability of detecting bad data. Assuming that the network parameters are known and the measurement devices are correctly calibrated, the main source of bad data is topological errors in the model. In this dissertation, we propose a methodology for robust power system state estimation (PSSE) modeled by AC power flow equations when there exists a small number of topological errors. The developed technique utilizes the availability of a large number of SCADA measurements and minimizes the L-1 norm of nonconvex residuals augmented by a nonlinear, but convex, regularizer. Representing the power network by a graph, we first study the properties of the solution obtained from the proposed NLAV estimator and demonstrate that, under mild conditions, this solution identifies a small subgraph of the network that contains the topological errors in the model used for the state estimation problem. Then, we introduce a method that can efficiently detect the topological errors by searching over the identified subgraph. In addition, we develop a theoretical upper bound on the state estimation error to guarantee the accuracy of the proposed state estimation technique. The power flow equations are nonlinear, and may admit multiple solutions. In the past, the conventional wisdom was to assume that the solution becomes unique by restricting it to "realistic'' or "physically realizable'' values. However, various examples in the literature show that multiple solutions may persist even after restricting either voltage magnitudes or phase angle differences to "physically realizable'' values. This dissertation establishes sufficient conditions for the uniqueness of AC power flow solutions via the monotonic relationship between real power flow and the phase angle difference. More specifically, we prove that the P-Theta power flow problem has at most one solution for any acyclic or GSP graph. In addition, for arbitrary cyclic power networks, we show that multiple distinct solutions cannot exist under the assumption that angle differences across the lines are bounded by some limit related to the maximal girth of the network. We also introduce a series-parallel operator and show that this operator obtains a reduced and easier-to-analyze model for the power system without changing the uniqueness of power flow solutions. In the next part of this dissertation, the above work is extended and we establish general necessary and sufficient conditions for the uniqueness of P-Theta power flow solutions in an AC power system using properties of the monotone regime and the power network topology. We show that the necessary and sufficient conditions can lead to tighter sufficient conditions for the uniqueness in several special cases. Our results are based on the previously introduced notion of maximal girth and a new notion of maximal eye. Moreover, we develop a series-parallel reduction method and search-based algorithms for computing the maximal eye and maximal girth, which are necessary for the uniqueness analysis. Reduction to a single line using the proposed reduction method is guaranteed for 2-vertex-connected Series-Parallel graphs.In the final part of this dissertation, we present a methodology based on homotopy to find the globally optimal solutions of nonconvex optimization problems. Optimal power flow (OPF) is a fundamental problem in power systems analysis for determining the steady-state operating point of a power network that minimizes the generation cost. In anticipation of component failures, such as transmission line or generator outages, it is also important to find optimal corrective actions for the power flow distribution over the network. The problem of finding these post-contingency solutions to the OPF problem is challenging due to the nonconvexity of the power flow equations and the large number of contingency cases in practice. In this paper, we introduce a homotopy method to solve for the post-contingency actions, which involves a series of intermediate optimization problems that gradually transform the original OPF problem into each contingency-OPF problem. We show that given a global solution to the original OPF problem, a global solution to the contingency problem can be obtained using this homotopy method, under some conditions. With simulations on Polish and other European networks, we demonstrate that the effectiveness of the proposed homotopy method is dependent on the choice of the homotopy path and that homotopy yields an improved solution in many cases.

For over 30 years we have been negotiating agreements that try to reduce greenhouse gas emissions. The aim is to stabilize their concentration in the atmosphere at a level that would prevent dangerous anthropogenic interference with the climate system. In 2018, the Intergovernmental Panel on Climate Change stated that the 1.5°C goal could be achieved if the electricity sector would become net zero emissions by 2050. Different countries have been pushing the frontier to reduce their emissions by deploying renewable energy sources (RES). Despite these efforts, we still have a long way to go. Research on how and when to install and how to operate more RES in power systems needs to continue advancing as we aim to reach higher levels of penetration. In addition, academics need to translate and communicate these findings to policy makers. The contributions of this dissertation in this field have been to: 1.propose a new time-varying representation for power dynamics that reflects the presence of RES,2.design through machine learning a stable time-invariant frequency controller for the new time-varying power dynamics,3.explore the trade-off between information availability to the frequency control agents and their performance and stability,4.show the cost effectiveness of stronger RES targets in the U.S. by 2030 given the carbon reductions goals of 2050, and5.model climate change uncertainty through a stochastic formulation of the capacity expansion of power systems in the U.S. with high penetration of RES.As more non-synchronous RES participate in power systems, the system's inertia decreases and becomes time dependent, challenging the ability of existing control schemes to maintain frequency stability. System operators, research laboratories, and academic institutes have expressed the importance to adapt to this new power system paradigm. However, power dynamics have been modeled as time-invariant, by not modeling the variability in the system's inertia. To address this, we propose a new modeling framework for power system dynamics to simulate a time-varying evolution of rotational inertia coefficients in a network. Power dynamics are modeled as a hybrid system with discrete modes representing different rotational inertia regimes of the network.Using this new hybrid model for power dynamics, we present a framework to design a fixed learned controller based on datasets of optimal time-varying LQR controllers. We test the performance of the controller in a twelve-bus system. By adding virtual inertia we can guarantee stability of high-renewable (low-inertia) modes. The novelty of our work is to propose a design framework for a stable controller with fixed gains for time-varying power dynamics. This is relevant because it would be simpler to implement a proportional controller with fixed gains compared to a time-varying control. To expand this work, we introduce a framework to learn sparse time-invariant frequency controllers in a power system network with a time-varying evolution of rotational inertia. We design a controller that uses as features the system’s states. In other words, we design a control proportional to the angles and frequencies. Virtual inertia is included in the controllers to ensure stability. One of the findings is that it is possible to restrict communication between the nodes by reducing the number of features in the controller (from 22 to 10 in our case study) without disrupting performance and stability. Furthermore, once communication between nodes has reached a threshold, increasing it beyond this threshold does not improve performance or stability. There is a correlation between optimal feature selection in sparse controllers and the topology of the network.In the second part of this dissertation we study the cost and lock in of carbon intensive technologies due to weak medium-term policies. We use SWITCH WECC-- a power system capacity expansion optimization model with high temporal and geographical resolution. We test three carbon cap scenarios. For each scenario, we optimize the power system for a medium timeframe (2030) and a long timeframe (2050). In the medium timeframe optimizations, by 2030 coal replaces gas power. This occurs because the long optimization foresees the stronger carbon cap in 2050. Therefore, it is optimal to transition towards cleaner technologies as early as 2030. The medium-term optimization has higher costs in 2040 and 2050 compared to the long optimization. Therefore, to minimize total costs to reduce emissions by 80% in 2050, we should optimize until 2050 or have stronger carbon cap policies by 2030 (such as a 26% of emissions reductions from 1990 levels by across the WECC).Typical electricity-grid capacity expansion models make investment decisions with fixed inputs (e.g., fixed electricity demands and hydro-power availability). The resultant electricity supply system may not be robust to future climate change-driven uncertainties in energy demand and supply. We present the first climate change stochastic long-term (2050) capacity expansion and operation electricity grid model for the Western North America electricity region, with high temporal and spatial resolution. The Stochastic SWITCH WECC model generates a least cost portfolio of power plants that is robust to varying future climate conditions using a multi-stage optimization approach with varying electricity-demand and hydropower-availability inputs under three climate change scenarios. Results show that an optimal robust electricity supply portfolio in the WECC for 2050 has about 4% higher overall installed capacity than the average mix of the three scenarios modeled separately, and about 5.6% higher installed gas capacity, due to the greater need for operational flexibility under the wider range of possible conditions.

This dissertation describes tools and techniques for applying data to inform decisions made in electric power systems. We explore how limitations inherent to data sets can contribute to model uncertainty, and formulate quantitative methods for issuing recommendations that are robust to uncertainty. The work provides a basis for justifying advancements in data collection and monitoring to increase confidence in decision-making outcomes.Data about the past can offer valuable insights about the present and future. Data can improve the fidelity of simulations for characterizing performance, and can shed light on emerging risks. Data offers a basis to validate performance, to inform targeted maintenance, and to predict the outcome of candidate decisions.Though data can be a powerful tool, our interpretations of data and even the data themselves are subject to error. Data analysts must judge whether the data are sufficient to answer the question at hand, whether the questions themselves are well-posed, or whether more suitable data should be sought after. The current work describes methods to inform these judgements by exploring how they could alter interpretations of risk and performance, contextualizing these different interpretations in terms of recommendations issued to decision-makers.The work centers around three case studies that apply imperfect data to inform such recommendations. We consider data sets where relevant information is lost, where observational history is sparse, and where spatial coverage is limited. We synthesize data using exploratory data analysis, hierarchical modeling, and Bayesian optimization. These methods internalize what is known about the limitations of underlying data sets to examine uncertainty in decision-making outcomes. Each case study formulates a quantitative decision-support model, and offers a quantitative basis to inform the collection of new information that could help decision-makers to act with greater confidence.

Fast growing and emerging economies face the dual challenge of sustainably expanding and improving their energy supply and reliability while at the same time reducing poverty. Critical to such transformation is the provision of affordable and sustainable electricity for the 1 billion peo-ple that do not have access and hundreds of millions connected to underperforming systems. However, expansion of electricity service in emerging economies is following the same century old paradigm under which power systems evolved: large central generation plants connected to load centers through a transmission grid and distribution lines with radial flows. This paradigm is now being challenged by the development and diffusion of modular generation and storage technologies. This dissertation contributes with models and results to help policy makers and pri-vate developers embrace these technological changes to develop cost-effective strategies to ex-pand electricity access.I use the SWITCH capacity expansion model to explore low carbon development pathways for the Kenyan electricity generation and transmission sectors under a set of plausible scenarios for fast growing economies that include uncertainty in load projections, capital costs, operational performance, and technology and environmental policies. This research investigates the genera-tion and transmission costs and operational and environmental impacts on the Kenyan expansion pathway of these variables and policies. I find that the Kenyan power system presents a unique transition from one basal renewable resource – hydropower – to another based on geothermal and wind power for ~90% of total capacity. I also find that a cost-effective and viable suite of solu-tions includes availability of storage, diesel engines, and transmission expansion to provide flexi-bility to enable up to 50% of wind power penetration. Results suggest that fast growing and emerging economies could benefit by incentivizing anticipated strategic transmission expansion. “Zero carbon emission” by 2030 pathways are possible with only moderate levelized cost increas-es of between $3 to $7/MWh with a number of social and reliability benefits.Traditional capacity expansion modeling that focuses on large-scale generation and transmis-sion does not evaluate the potential contribution of distributed resources – modular technologies that can be deployed close to load centers. To improve on these modeling limitations, I use a novel approach to assess the sequencing and pacing of centralized, distributed, and off-grid elec-trification strategies by developing and employing the Grid and Access Planning (GAP) model. GAP is a capacity expansion model to jointly assess operation and investment in utility-scale generation, transmission, distribution, and demand side resources. Contrary to the current prac-tice, I find hybrid systems that pair grid connections with distributed energy resources (DER) are the preferred mode of electricity supply for greenfield expansion under conservative reductions in PV and energy storage prices. I also find that when distributed PV and storage are employed in power system expansion, there are savings of 15%-20% mostly in capital deferment and re-duced diesel use. Results show that enhanced financing mechanisms for DER PV and storage could enable 50-60% of additional deployment and save 15 $/MWh in system costs. These re-sults have important implications to reform current utility business models in developed power systems and to guide development of electrification strategies in underdeveloped grids.A comprehensive development of electrification strategies requires complementing modeling results with empirical observations of the electrification process. I leverage two household budget surveys developed in Kenya in 2006 and 2016 as a unique data-driven window into the drivers of electricity access and the evolution of the electrification process. I find evidence that gains in electrification have come from grid extensions into rural areas as well as from people migrating from rural areas to cities, that rural grid extensions may be underutilized, and that poorer quintiles remain vastly unsupplied. Results highlight the role that modular technologies can play to com-plement traditional grid extensions to increase the pace and coverage of electrification efforts. Ultimately, comprehensive electrification strategies will need to integrate technological advances, new modeling paradigms, and the domestic socio-cultural context into policy making to advance electricity access decisively in low-income economies.

There has been multiple national directives to enhance the economic operations of the power transmission system and promote the efficient use of the current grid configuration and resources. Transmission switching has been proposed as a new control method for various benefits, including improving the economic efficiency and meeting the reliability requirements. Optimal Transmission Switching model has been introduced to find an optimal generation dispatch and network topology to minimize the dispatch cost. Binary decision variables are used to denote the control of the transmission lines, which makes the model a nonlinear program. It suffers from curse of dimensionality and faces serious computational challenges.To tackle the computational challenge of the Optimal Transmission Switching model, we propose three greedy algorithms in which only one line is switched at an iteration. In each iteration we solve a series of linear programs or smaller MIP programs, which can be implemented in parallel with the aid of high performance computing. The first algorithm enumerates all the possible line switching actions. The second algorithm produces a priority list ranking lines by a sensitivity factor based on dual criterion, and evaluate lines starting from the top of the list. The last one divides lines into small groups and consider each group at one time. We test the algorithms on the IEEE 118-Bus network and the FERC 13,867-Bus network which is representative of PJM Regional Transmission Organization. The results show that all three proposed algorithms result in cost reduction close to the best known optimal within a reasonable timeframe for IEEE network. For the FERC network which can not be solved directly by the OTS, the first two greedy algorithms are able to produce switching sequences which result in considerable cost reduction.Furthermore we propose three machine learning based methods to produce priority lists for ranking possible line switching actions to facilitate faster searching. The algorithms take in the parameters from network status and network configuration to produce a standardized score representing the possible cost reduction of the line switching action. The numerical results on IEEE network and FERC network are presented. We evaluate the effectiveness of the priority lists based on dual criterion and machine learning methods by both regression analysis and comparison with random lists. Based on the algorithms in literature and those we develop, we propose an algorithm selection method which selects the algorithm to optimize the cost improvement at each iteration. They show improvement in the cost reduction compared to the individual algorithms, especially for the FERC network.

Contemporary power systems are characterized by increasing penetrations of renewable and distributed energy resources (DERs), which present both opportunities and challenges for reliable system operation. Uncertainty in renewable generation must be managed over both shorter and longer timescales, necessitating greater flexibility in the supply and demand of electricity. Fortunately, this rise in renewables has been accompanied by the growth of flexible energy resources such as energy storage and aggregated demand flexibility. However, these assets are generally not adequately integrated into existing electricity market structures leading to many benefits of their flexibility going unrealized. Additionally, distributed energy resources, such as rooftop solar and electric vehicles, present a challenge to the traditional top-down management of power systems, requiring new decentralized control and market architectures to ensure their safe operation and to maximize their value. This dissertation tackles a range of questions relating to the integration of flexible and distributed resources into electricity markets in the presence of uncertainty, using techniques from control theory, game theory, multiparametric programming, and distributed optimization.The main contributions of this dissertation encompass novel analysis and design of markets, both centralized and decentralized, that explicitly integrate distributed and flexible resources, numerical methods providing exact solutions to multi-leader follower equilibrium problems among strategic bidders, and insights into the role of virtual bidders in managing demand uncertainty in two-stage markets. Chapter 1 introduces the context for the dissertation, describing trends driving the energy industry and the challenges and opportunities they pose. Current practice and existing literature are described with regard to uncertainty, flexibility, and decentralized markets and control. Chapter 2 lays out the mathematical background for the dissertation, describing models for power flow, electricity markets, and market participants, in addition to the theory of multiparametric programming. Chapter 3 demonstrates the equivalence of virtual bidding and stochastic optimization in two-stage markets under certain conditions. It also characterizes the equilibrium of a crowd of virtual bidders with heterogeneous beliefs and explores simple learning strategies for virtual bidders to reach equilibrium. Chapter 4 studies robust Cournot-Bertrand equilibria among generation firms. Firms anticipate demand and renewable uncertainty and optimize their actions to ensure their profits are robust to this uncertainty. The chapter concludes by discussing the effects on market outcomes of this behavior. Chapter 5 addresses the more general topic of strategic bidding by generation firms in electricity markets. This question involves solving non-convex bilevel optimization problems, which form part of the wider class of multi-leader follower problems. Both the single firm problem, constituting an MPEC, and the multi-firm equilibrium problem, constituting an EPEC, are solved exactly using techniques from multiparametric programming. This latter contribution, in particular, represents a significant advance over current methods, generalizing the solution of supply function equilibrium problems to cases with asymmetric players and transmission constraints. Chapter 6 analyzes a number of regulatory models for the integration of energy storage into wholesale electricity markets, including Open Access Storage, and investigates the implications and incentives of a variety of storage ownership structures under each model. Chapter 7 demonstrates the potential of load flexibility for congestion relief, defining the notion of a congestion-free dispatch. Chapter 8 describes the design of a decentralized coordination mechanism for DERs in a microgrid using blockchain technology. The implementation uses ADMM with a smart contract on the blockchain serving as the ADMM coordinator. Chapter 9 illustrates the design of an open-gate forward market with specific flexibility products for DERs, allowing users to hedge better against uncertainty and correctly express and maximize the value of their flexibility. A number of algorithms are presented to address the combinatorial scheduling and pricing problem involved with flexible loads.

Demand Response (DR) serves to reduce the demand for electricity especially during times when supply is scarce or expensive. The program entails calling the recruited consumers to reduce their energy consumption from an established baseline. Baseline is an estimate of the counter-factual against which load reductions are measured to determine payments to consumers. This creates an incentive for consumers to inflate their baseline so that their payments are inflated. There have been reported cases where consumers have inflated their baseline to increase their payments. To address this, we propose a novel self-reported baseline mechanism for DR programs in both retail and wholesale electricity markets. Also by comparing with mechanisms used in practice by System Operator's like CAISO (California Independent System Operator) we show that the proposed mechanism is either reliable or requires less payments for the same level of load curtailment provided by consumers.Finally we consider a repeated DR setting where the DR events repeat. We propose a baseline DR mechanism and a pricing policy that learns consumer's response to payments and exploits the learnt information such that the cumulative losses in savings, compared to the oracle determined optimal pricing mechanism, does not grow at an uncontrolled rate. In particular we show that it grows at a sub-linear rate.

This dissertation discusses the design and validation of a community-scale DC microgrid architecture that can integrate renewable generation sources by using a distributed control scheme implemented on the microgrid-connected households to mitigate resource variability. The presented system is designed to address the technical and economic challenges of electrifying emerging regions. The levelized cost of electricity (LCOE) is expected to be less than $0.40 per kW-hr over a 15 year horizon. Solar panel costs have steadily declined to make them a very affordable option for distributed generation. However, battery costs have not declined at the same rate. The important figure of merit for microgrids is the efficiency of stored electricity. The architecture presented in this work is shown to minimize losses in stored electricity. Another key feature of the system is the ability to prioritize loads by controlling the power flow of the system. There are three key components of the presented DC microgrid architecture: source converters, fanout nodes, and household power management units (PMUs). The functionality of each of these components and the rationale for their design choices is presented in detail. The PMUs implement a distributed loadline control scheme to mitigate variability in available grid power. The design of the distributed control scheme of the DC microgrid is discussed in detail. The relevance of Lyapunov based analysis towards the large signal stability of grid voltage is also discussed.The first-generation system uses a phase shifted full-bridge converter topology in the PMUs to convert from 380-to-12V in a single stage. Analytical analysis of this topology shows large signal and small signal stability. Transient response of the grid voltages and currents is shown through simulations. Experimental results are presented that showcase the operation and efficiency of system hardware.The second-generation system incorporates a two-stage conversion architecture. The first stage is a 380-to-48V fixed ratio bus converter, which is used to generate a 48V bus for a local cluster of connected households. In the second stage, PMUs utilize a buck converter topology to step down from 48V-to-12V in each household. In addition to simulation and experimental validation of the hardware, a cost analysis is also presented to determine LCOE of system. The second-generation architecture achieves higher efficiency and reduced cost in comparison to the first-generation. Lastly, grounding and protection of the microgrid is discussed in detail. Relevant examples from existing distribution schemes are presented and the rationale behind existing schemes are explored. A grounding and protection scheme that is suited to this dissertation's DC microgrid is presented. The protection scheme is validated throgh relevant simulations and experiments.This work presents a grid topology that is designed to adapt to a changing landscape of solar PV prices and meets the challenge of providing a scalable and low-cost electrification solution. Future efforts in addressing the energy needs of unelectrified regions can build on the work presented in this dissertation.

Power systems are part of the nation’s critical infrastructure and they support several indispensable services of our civilization such as hospitals, transportation systems, and telecommunications. Among the many requirements that power systems need to satisfy, power systems need to ensure that voltages and currents in the power grid are sinusoidal with a synchronous frequency of 50 or 60 Hz. Failure to do so would cause damage in appliances as well as in electrical industrial machinery that were developed under the assumption of sinusoidal voltages and currents with constant frequency. Furthermore, according to the North American Electric Reliability Corporation, frequency divergence of one or more of the generators that supply power to the grid, i.e., loss of synchronization, can lead to vibrations causing serious damage to the generators. As documented in the United States Department of Energy report on the August 14, 2003 blackout in Canada and the Northeast of the United States, frequency swings are the main reason for blackouts to spread across power systems. This makes the preservation of synchronization of generator frequencies one of the most important problems in power systems. This problem is also known as the transient stability problem in the power systems literature.The classical models used to study the transient stability problem implicitly assume that all the generators are rotating at angular velocities close to the synchronous frequency. This assumption is known not to hold in real power systems. A well documented example by the Department of Energy is the final stage of the August 14, 2003 blackout. This makes us question the validity of the existing tools and methods, based on classical assumptions and models, to predict and prevent the spread of blackouts.In this work, we abandon the classical models and replace them with energy-based models derived from first principles that are not subject to hard-to-justify classical assumptions. In addition to eliminate assumptions that are known not to be satisfied, we derive intuitive conditions ensuring the transient stability of power systems. Providing such conditions in theclassical framework with lossy transmission lines is a problem that has remained unsolved for more than sixty years and partial solutions under very restrictive assumptions have only recently been found. This is to be contrasted with the conditions described in this thesis that naturally handle lossy transmission lines. With the help of the insights we gained in the analysis performed in Section 4.3, we design easy-to-implement controllers that solve the transient stability problem in power systems. We also provide a novel way of performing circuit reduction, aiming to reduce the complexity of transmission grid models. Kron reduction, which is performed under steady state assumptions, is the standard circuit reduction technique used in the power systems literature. The novel circuit reduction method described in Section 3.1 shows how to perform Kron reduction for a class of electrical networks without these steady state assumptions.

Key to a sustainable future is the transformative human use of energy and transportation--from a fossil-fuel dominant to a renewables-mixed portfolio of energy production, from a supply-follow-demand to a demand-responsive pattern of power consumption, and from capacity-oriented to usage-based allocation of logistics mobility. Motivated by these trends, my dissertation presents three essays to address the challenges that governments and businesses worldwide face in, respectively, 1) planning infrastructure for wind energy production, 2) designing coordination strategies for large-scale charging of plug-in electric vehicles, and 3) evaluating the economic and environmental viability of using shared mobility for retail e-commerce. Two common threads underlying my dissertation research are 1) strategic decision-making with insights into the operational level and 2) network optimization that takes into account the interdependencies both within and out of system boundaries. Solving these network optimization problems invokes the techniques of mixed-integer conic programming, decomposition algorithms, and continuous approximation modeling. Specifically, the three essays are organized as three chapters of the dissertation:Chapter 1 studies the problem of jointly planning energy storage (ES) and transmission for wind energy generation. Regions with abundant wind resources usually have no ready access to the existing electric grid. However, building transmission lines that instantaneously deliver all geographically distributed wind energy can be costly. Energy storage systems can help reduce the cost of bridging wind farms and grids, and can mitigate the intermittency of wind outputs. We propose models of transmission network planning with colocation of ES systems. Our models determine the sizes and sites of ES systems as well as the associated topology and capacity of the transmission network under the feed-in-tariff policy instrument. We first formulate a location model as a mixed-integer second-order-conic program to solve for the ES-transmission network design with uncapacitated storage. Then we propose a method to choose ES sizes by deriving a closed-form upper bound. The major insight is that, in most cases, using even small-sized ES systems can significantly reduce the total expected cost, but their marginal values diminish faster than those of the transmission lines as their capacities expand. Despite uncertainties in climate, technologies, and construction costs, the cost-efficient infrastructure layout is remarkably robust. We also identify the major bottleneck cost factors for different forms of ES technologies.Chapter 2 presents a hierarchical optimal control framework to coordinate the charging of plug-in electric vehicles in multifamily dwellings. A particular scenario is considered where distributed urban residential communities access electric power supplies through a common primary distribution transformer. We first formulate a centralized finite-horizon control problem. The proposed multistage mixed integer program seeks to maximize the total utility of the charging service provider while satisfying customers' charge demands and transformer capacity constraints. By exploiting the structure of the centralized model, we decompose the centralized problem with respect to each parking deck, based on the Lagrangian relaxation method; we design an effective heuristic method to find feasible solutions to speed up convergence. Case studies on operations of five parking decks following different charging strategies are carried out. Simulation results demonstrate that the proposed distributed hierarchical charging strategy outperforms the centralized charging strategy from the perspective of computational requirements. System reliability and customer privacy protection are also discussed.Chapter 3 studies an integrated logistics system with shared mobility for retail e-commerce. Two socioeconomic transformations, namely, the booms in sharing economy and retail e-commerce, lead to the prospect where shared mobility of passenger cars prevails throughout urban areas for home delivery services. Local governments and logistics services providers are in need of evaluating the potentially substantial impacts of this mode shift, given their societal and environmental concerns and economic objectives. We addresses this need by providing a logistics planning framework: 1) Part one presents logistics planning models. These models characterize optimal routes of short-haul trucks and passenger cars, and generate the optimal density of service zones within which passenger vehicles pick up goods and fulfill the last-mile delivery. 2) Part two, based on empirical estimates, analyzes operating costs and greenhouse gas emissions implications of this sharing logistics paradigm. The findings suggest that a transition to this paradigm has the potential for creating considerable economic and environmental benefits, although immediate savings are not as achievable as one may conjecture. If being in this paradigm, even exclusively minimizing operating costs does not significantly increase emissions relative to the minimum level of emissions. A non-linear payment scheme can be used to efficiently induce shared mobility into passenger car home delivery services.

This dissertation is concerned with the design of branch-and-cut algorithms for a variety of nonconvex nonlinear problems pertaining to power systems operations and planning. By understanding the structure of specific problems, we can leverage powerful commercial optimization solvers designed for convex optimization and mixed-integer programs. The bulk of the work concerns the Alternating Current Optimal Power Flow (ACOPF) problem. The ACOPF problem is to find a minimum cost generation dispatch that will yield flows that can satisfy demand as well as various engineering constraints. A standard formulation can be posed as a nonconvex Quadratically Constrained Quadratic Program with complex variables. We develop a novel spatial branch-and-bound algorithm for generic nonconvex QCQP with bounded complex variables that relies on a semidefinite programming (SDP) relaxation strengthened with linear valid inequalities. ACOPF-specific domain reduction or bound tightening techniques are also introduced to improve the algorithm's convergence rate. We also introduce second-order conic valid inequalities so that any SDP can be outer-approximated with conic quadratic cuts and test the technique on ACOPF. Another application is the incorporation of convex quadratic costs in unit commitment, which is a multi-period electric generation scheduling problem. We show that conic reformulation can both theoretically and practically improve performance on this mixed-integer nonlinear problem. We conclude with methods for approximating a mixed-integer convex exponential constraint. Applications include capital budgeting, the system reliability redundancy problem, and feature subset selection for logistic regression.

Microgrids, as one of the key components to enable the future smart grid, refer to low-voltage power distribution systems integrated with distributed energy resources (DERs) and controllable loads, which can operate either with or without the grid (i.e., grid-connected or islanded mode). The integration of DERs and controllable loads brings tremendous opportunities to increase power system efficiency, sustainability, and reliability. However, the intermittency and variability of renewable DERs and limited supply especially when the microgrid is operating in islanded mode introduce significant challenges to maintain the fundamental supply-demand balance for system stability. Therefore, the goal of this dissertation is to solve the supply-demand balancing problem in microgrids using optimization-based energy management.Most of the existing energy management algorithms in the literature consider the aggregate supply-demand balance as an abstract mathematical function while omitting the underlying power distribution network and the associated power flow and system operational constraints. Consequently, such approaches may result in control decisions that violate the real-world constraints. Therefore, in the first part of this dissertation, we study the supply-demand balancing problem in microgrids under more realistic conditions and propose algorithms for microgrid energy management that take into account the power flow and system operational constraints on a distribution network. By incorporating the distribution network in the modeling, we present the relationship between the physical structure of a microgrid and the energy management on the network.Another major challenge in microgrid energy management is to design a two-way communication system in order to implement the algorithms. A variety of heterogeneous devices in a microgrid need to be managed by such a system using the energy management algorithms. Unfortunately, most of those devices still use proprietary protocols and cannot interoperate with each other. Furthermore, many devices managed by the system reside on the customer side requiring autonomy and local intelligence. Therefore, in the second part of this dissertation, we focus on the design and implementation of a system architecture that enables interoperability and autonomy for microgrid energy management. We present the design of a unified communication interface that is protocol and technology agnostic for interoperability and a decentralized system architecture for autonomy on the customer side.

Electrical power is the bedrock of modern civilization, and the large-scale hierarchical structure of bulk generation, transmission, and distribution has served us well for more than one hundred years. Currently however, the landscape of energy production is shifting, as economic, environmental, and technological factors are pushing power generation towards a future dominated by distributed generation from renewable energy sources. While grid-wide control strategies and architectures in bulk power systems were designed for slow time-scales and large synchronous generators, renewable energy interfaced through power electronic inverters allows for rapid response and greater flexibility in both local controller design and grid-wide control architectures.This thesis focuses on exploring the limitations of both local controller design and control architectures for inverter-based power grids. Our contributions can be broadly divided into two categories. First, we study the classic primary droop controllers proposed for inverter-based power grids, and provide the first nonlinear analysis of the closed-loop frequency and voltage dynamics resulting from the controllers. We present tight analytic conditions for the existence and uniqueness of stable equilibrium points, thereby quantifying the fundamental limits of these controllers. In the second portion of the thesis, we propose and analyze a distributed control architecture which takes the place of the classical centralized secondary control layer. The distributed controllers combine classic droop ideas from power systems with agreement algorithms studied in cooperative control and multi-agent systems, leading to a scalable control architecture which achieves centralized performance. The algorithms require only sparse communication between nearby inverters in the power grid, and require no a priori knowledge of the grid topology or load demands. We present extensive analysis results, along with experimental results validating our designs, and outline directions for future research.

Deep de-carbonization of the electric power sector is indispensable to achieving climate change mitigation. This work explores how aggressive reductions in electricity sector emission levels can be achieved, what the associated costs would be, and how these costs may be minimized. Integrating increased levels of intermittent renewable energy sources into the electricity grid poses new challenges to system planning, operation, and reliability, increasing the need for models that can merge the capabilities of capacity-expansion and production cost simulation models.I describe the operational detail I have incorporated into the long-term investment framework of the SWITCH model to allow for more accurate evaluation of both the potential contribution of intermittent renewable technologies to electricity decarbonization and the associated system flexibility requirements. I have implemented a series of enhancements to the model's treatment of system operations and generator types -- the SWITCH "system flexibility module" -- in order to simulate unit commitment as realistically as possible, at an unprecedented resolution for a capacity-expansion model of a large geographic area, offering some of the most detailed treatment to date of day-to-day system operations in an investment model.I run a range of scenarios to explore the effect of various sources of uncertainty for system development between present day and 2050 in the Western Electricity Coordinating Council (WECC). I include sensitivities for technology costs, fuel prices, technology availability, demand profile, and availability and cost of system flexibility options. I find that meeting a carbon emissions reduction target of 85 percent below 2050 levels is feasible across a range of assumptions. The cost of achieving the goal is highly uncertain, but a number of opportunities to contain costs exist. In the 2030 timeframe, lowering the cost of solar technologies to the SunShot target is the main cost-reduction strategy. Achieving the ARPA-E battery cost target has a small impact on system costs through 2030, as other sources of flexibility are available to the system, including gas generation, hydro, and CAES. The price of natural gas is key to its utilization in the 2030 timeframe, but is not an important driver in 2050 when natural gas flexibility is of high value yet fuel use is limited by the carbon cap.Solar PV deployment is the main driver of CAES and battery storage deployment: its diurnal periodicity results in opportunities for daily arbitrage that these technologies are well suited to provide. Storage operation is very different from present day patterns - storage tends to charge during the day when solar PV is available and discharge in the evening and at night. Similarly, the ability to shift loads to the daytime solar peak could have cost-reduction benefits for the system. Wind output exhibits large seasonal variations; because it can remain at very low (or very high) levels for extended periods of time, it does not benefit from CAES and battery storage (operating as providers of daily arbitrage) as much as solar PV does, and instead requires storage with a large energy subcomponent.CSP with thermal storage is an important component of the 2050 power system, but it directly competes with the combination of solar PV and batteries. If low solar PV costs and low battery costs are achieved, the two technologies may be deployed at large-scale, displacing CSP with thermal storage. The combination of SunShot solar technology and advanced battery technology has the largest impact on total storage capacity deployment in 2050. This combination can provide substantial savings through 2050, greatly mitigating the cost of climate change mitigation and outperforming the nuclear-dominated scenario given the costs assumed here.Policy goals for storage deployment should incorporate both the power subsystem component and the energy subsystem component of energy storage. In addition, storage deployment requirements should be set as part of overall system development goals as system flexibility needs will vary depending on the rest of the grid mix. Policy ought to be technology-neutral and support a comprehensive portfolio of system flexibility options, allowing flexible generation, demand response, and flexible electric vehicle charging, which can provide comparable services, to compete with storage on a level playing field. The increase in system flexibility requirements can be managed through a system-wide approach including regional cooperation to strategically plan for transmission interconnection and geographic diversity of renewable resource deployment to mitigate the variability of overall output. System-level planning is critical to ensure that appropriate incentives are put in place for all grid assets to fully recover their costs and justify investment while providing the most value to the system and ensuring cost-effective system development over time.

The electric power system is undergoing dramatic transformations due to the emergence of renewable resources. However, integrating these resources into the electric grid has proven to be difficult for two main reasons: these resources are uncertain and distributed. This thesis discusses how uncertainty and distributedness of the new resources are manifested as challenges in time and spatial scales, and how we can optimally control the power grid to overcome these challenges. The theme of this thesis is that in order to control the new resources, we need a better understanding of the physical power flow in the system. To illustrate the spatial-scale challenge, we consider the voltage regulation problem for distribution networks. With a deep penetration of distributed energy resources the voltage magnitudes in a distribution system can fluctuate significantly. To control these resources such that voltage profiles remain flat, we need to coordinate the tens of thousands of households in the distribution network. By studying the geometry of power flows in the network, we show that even though the voltage regulation problem is non-convex, it can be convexified exactly through a semidefinite relaxation . Based on this insight, we an optimal and decentralized algorithm for this problem. Only communication between electrical neighbors are required in the algorithm, thus allowing it to scale to problem of large sizes. The second problem we consider is dispatching generators in the transmission network under deep penetration of wind power. Because of on/off and ramp-rate constraints, traditional generators need to be dispatched before the actual time of delivery of energy. On the other hand, wind power cannot be dispatched and is difficult to predict in advance. This creates a mismatch in time-scale between the controls (traditional generators) and the randomness in the system (wind). We capture this challenge as a two-stage stochastic dispatch problem. In contrast to the standard Monte Carlo solutions, we show that the dimensional of the problem can be reduced dramatically by using forecast informations. This dimensional reduction is based on a better understanding of congested DC optimal power flow problems. Using this reduction, we show how to calculate the optimal reserve margin for the system in the presence of renewables, and quantify the intrinsic impact of uncertainties on the system cost.

While the traditional goal of an electric power system has been to control supply to fulfill demand, the demand-side can plan an active role in power systems via Demand Response (DR), defined by the Department of Energy as ``a tariff or program established to motivate changes in electric use by end-use customers in response to changes in the price of electricity over time, or to give incentive payments designed to induce lower electricity use at times of high market prices or when grid reliability is jeopardized." DR can provide a variety of benefits including reducing peak electric loads when the power system is stressed and fast timescale energy balancing. Therefore, DR can improve grid reliability and reduce wholesale energy prices and their volatility.This dissertation focuses on analyzing both recent and emerging DR paradigms. Recent DR programs have focused on peak load reduction in commercial buildings and industrial facilities (C&I facilities). We present methods for using 15-minute-interval electric load data, commonly available from C&I facilities, to help building managers understand building energy consumption and `ask the right questions' to discover opportunities for DR. Additionally, we present a regression-based model of whole building electric load, i.e., a baseline model, which allows us to quantify DR performance. We use this baseline model to understand the performance of 38 C&I facilities participating in an automated dynamic pricing DR program in California. In this program, facilities are expected to exhibit the same response each DR event. We find that baseline model error makes it difficult to precisely quantify changes in electricity consumption and understand if C&I facilities exhibit event-to-event variability in their response to DR signals. Therefore, we present a method to compute baseline model error and a metric to determine how much observed DR variability results from baseline model error rather than real variability in response. We find that, in general, baseline model error is large. Though some facilities exhibit real DR variability, most observed variability results from baseline model error. In some cases, however, aggregations of C&I facilities exhibit real DR variability, which could create challenges for power system operation. These results have implications for DR program design and deployment.Emerging DR paradigms focus on faster timescale DR. Here, we investigate methods to coordinate aggregations of residential thermostatically controlled loads (TCLs), including air conditioners and refrigerators, to manage frequency and energy imbalances in power systems. We focus on opportunities to centrally control loads with high accuracy but low requirements for sensing and communications infrastructure. Specifically, we compare cases when measured load state information (e.g., power consumption and temperature) is 1) available in real time; 2) available, but not in real time; and 3) not available. We develop Markov Chain models to describe the temperature state evolution of heterogeneous populations of TCLs, and use Kalman filtering for both state and joint parameter/state estimation. We present a look-ahead proportional controller to broadcast control signals to all TCLs, which always remain in their temperature dead-band. Simulations indicate that it is possible to achieve power tracking RMS errors in the range of 0.26-9.3% of steady state aggregated power consumption. Results depend upon the information available for system identification, state estimation, and control. We find that, depending upon the performance required, TCLs may not need to provide state information to the central controller in real time or at all. We also estimate the size of the TCL potential resource; potential revenue from participation in markets; and break-even costs associated with deploying DR-enabling technologies. We find that current TCL energy storage capacity in California is 8-11 GWh, with refrigerators contributing the most. Annual revenues from participation in regulation vary from $10 to $220 per TCL per year depending upon the type of TCL and climate zone, while load following and arbitrage revenues are more modest at $2 to $35 per TCL per year. These results lead to a number of policy recommendations that will make it easier to engage residential loads in fast timescale DR.

The goal of residential load management is to manipulate the electricity demand of a group of consumers to match some desired shape. Effective load management needs two types of controls: systemic and local. Systemic control manipulates the aggregate power by signaling the consumers. Local control determines how the power consumer reacts to the signals. I present a price responsive intelligent thermostat for use in local load management. Further, I explore two types of systemic control using electricity price as the control variable. The first approach is to identify the system using a low order model and apply traditional control techniques. The second approach makes use of game theory to auction the electricity to the consumers. Both methods enable reliable and robust high resolution load management.