Search

The expansion of variable generation has driven a transition toward a 100\% non-fossil power system. New system needs are challenging system stability and suggesting the need for a redesign of the ancillary service (AS) markets. This paper presents a comprehensive and broad review for industrial practitioners and academic researchers regarding the challenges and potential solutions to accommodate high shares of variable renewable energy (VRE) generation levels. We detail the main drivers enabling the energy transition and facilitating the provision of ASs. A systematic review of the United States and European AS markets is conducted. We clearly organize the main ASs in a standard taxonomy, identifying current practices and initiatives to support the increasing VRE share. Furthermore, we envision the future of modern AS markets, proposing potential solutions for some remaining fundamental technical and market design challenges., Comment: 64 pages, 9 figures

This paper considers the investment coordination problem for the long term transmission capacity expansion in a situation where there are multiple regional Transmission Planners (TPs), each acting in order to maximize the utility in only its own region. In such a setting, any particular TP does not normally have any incentive to cooperate with the neighboring TP(s), although the optimal investment decision of each TP is contingent upon those of the neighboring TPs. A game-theoretic interaction among the TPs does not necessarily lead to this overall social optimum. We, therefore, introduce a social planner and call it the Transmission Planning Coordinator (TPC) whose goal is to attain the optimal possible social welfare for the bigger geographical region. In order to achieve this goal, this paper introduces a new incentive mechanism, based on distributed optimization theory. This incentive mechanism can be viewed as a set of rules of the transmission expansion investment coordination game, set by the social planner TPC, such that, even if the individual TPs act selfishly, it will still lead to the TPC's goal of attaining overall social optimum. Finally, the effectiveness of our approach is demonstrated through several simulation studies.

Pumped storage hydro units (PSHU) are great sources of flexibility in power systems. This is especially valuable in modern systems with increasing shares of intermittent renewable resources. However, the flexibility from PSHUs, particularly in the real-time market, has not been thoroughly studied. The storage optimization in a real-time market hasn't been well addressed. To enhance the use of PSH resources and leverage their flexibility, it is important to incorporate the uncertainties, properly address the risks and avoid increasing too much computational burdens in the real-time market operation. To provide a practical solution to the daily operation of a PSHU in a single day look-ahead commitment (LAC) and real-time market, this paper proposes two pumped storage hydro (PSH) models that only use probabilistic price forecast to incorporate uncertainties and manage risks in the LAC and real-time market operation. The price forecast scenarios are formulated only on PSHUs that minimizes the computational challenges to the Security Constrained Unit Commitment (SCUC) problem. Numerical studies in Mid-continent Independent System Operator (MISO) demonstrate that the proposed models improves market efficiency. Compared to traditional stochastic and robust unit commitment, the proposed methods only moderately increase the solving time from current practice of deterministic LAC. Probabilistic forecast for Real Time Locational Marginal Price (RT-LMP) on PSH locations is created and embedded into the proposed stochastic optimization model, an statistical robust approach is used to generate scenarios for reflecting the temporal inter-dependence of the LMP forecast uncertainties., Comment: 10 pages, 8 figures

This paper provides an enhanced modeling of the contingency response that collectively reflects high-fidelity physical and operational characteristics of power grids. Integrating active and reactive power contingency responses into the security-constrained optimal power flow (SCOPF) problem is challenging, due to the nonsmoothness and nonconvexity of feasible sets in consequence of piece-wise curves representing generator characteristics. We introduce a continuously-differentiable model using functions that closely resemble PV/PQ switching and the generator contingency response. These models enforce physical and operational limits by optimally allocating active power imbalances among available generators and deciding the bus type to switch from the PV type to the PQ type. The efficacy of this method is numerically validated on the IEEE 30-bus, 300-bus, and 118-bus systems with 12, 10, and 100 contingencies, respectively.

Pumped storage hydro units (PSHU) can provide flexibility to power systems. This becomes particularly valuable in recent years with the increasing shares of intermittent renewable resources. However, due to emphasis on thermal generation in the current market practices, the flexibility from PSHUs have not been fully explored and utilized. This paper proposes a configuration based pumped storage hydro (PSH) model for the day-ahead market, in order to enhance the use of PSH resources in the system. A strategic design of incorporating and fully optimizing PSHUs in the day-ahead market is presented. We show the compactness of the proposed model. Numerical studies are presented in an illustrative test system and the Midcontinent Independent System Operator (MISO) system.

As renewable energy penetration increases and system inertia levels drop, primary frequency control is becoming a critical concern in relatively small interconnections such as the Electric Reliability Council of Texas (ERCOT). To address this problem ERCOT is implementing a number of market rule changes including the introduction of a new Fast Frequency Response (FFR) reserve type to the electricity market. This FFR reserve type aims to help the traditional Primary Frequency Response (PFR) reserve type in arresting frequency decline in the event of a large generator outage. This paper derives reserve requirements to ensure sufficient reserve to arrest frequency decline before reaching the critical frequency threshold while coupling PFR reserve, FFR reserve, and system inertia. The general reserve requirement places limits on the amount of PFR reserve that can be provided by each unit based on its ramping capabilities. Two such limits are derived from first principles and another is proposed that is capable of accommodating the equivalency ratio introduced in previous work. These PFR reserve limits also provide first principles insight into equivalency ratios, which have only been studied empirically in the past. High-level insights are provided on a large Texas test case., Comment: 8 pages, 6 figures, 2 tables, submitted to IEEE Transactions on Power Systems

In this paper, we will consider the Look-Ahead Security Constrained Optimal Power Flow (LASCOPF) problem looking forward multiple dispatch intervals, in which the load demand varies over dispatch intervals according to some forecast. We will consider the base-case and several contingency scenarios in the upcoming as well as in the subsequent dispatch intervals. We will formulate and solve the problem in a Model Predictive Control (MPC) paradigm. We will present the Auxiliary Proximal Message Passing (APMP) algorithm to solve this problem, which is a bi-layered decomposition-coordination type distributed algorithm, consisting of an outer Auxiliary Problem Principle (APP) layer and an inner Proximal Message Passing (PMP) layer. The APP part of the algorithm distributes the computation across several dispatch intervals and the PMP part performs the distributed computation within each of the dispatch interval across different devices (i.e. generators, transmission lines, loads) and nodes or nets. We will demonstrate the effectiveness of our method with a series of numerical simulations., Comment: Accepted for publication in Elsevier ScienceDirect-International Journal of Electrical Power and Energy Systems

The operation of power grids is becoming increasingly data-centric. While the abundance of data could improve the efficiency of the system, it poses major reliability challenges. In particular, state estimation aims to learn the behavior of the network from data but an undetected attack on this problem could lead to a large-scale blackout. Nevertheless, understanding vulnerability of state estimation against cyber attacks has been hindered by the lack of tools studying the topological and data-analytic aspects of the network. Algorithmic robustness is of critical need to extract reliable information from abundant but untrusted grid data. We propose a robust state estimation framework that leverages network sparsity and data abundance. For a large-scale power grid, we quantify, analyze, and visualize the regions of the network prone to cyber attacks. We also propose an optimization-based graphical boundary defense mechanism to identify the border of the geographical area whose data has been manipulated. The proposed method does not allow a local attack to have a global effect on the data analysis of the entire network, which enhances the situational awareness of the grid especially in the face of adversity. The developed mathematical framework reveals key geometric and algebraic factors that can affect algorithmic robustness and is used to study the vulnerability of the U.S. power grid in this paper.

Application of Dynamic Mode Decomposition to clear-sky index forecasting of shadowing effects of convective fair-weather cumulus clouds is presented. Cloud dynamics are captured by sequences of visible-light photographic video frames. This method can be more easily applied to the modeling of cloud evolution than traditional fluid-based methods, and can enhance existing frozen-cloud advection methods. Its use is demonstrated for an actual fair-weather cumulus cloud image sequence and compared to an advection-only forecast. It is concluded that the method shows promise for very short-term clear-sky index forecasting for up to seven minute horizons., Comment: 20 pages, 10 figures

The roll-out of a flexible ramping product provides Independent System Operators (ISOs) with the ability to address ramping capacity shortages. ISOs procure flexible ramping capability by committing more generating units or reserving a certain amount of head room of committed units. In this paper, we raise the concern of the possibility that the procured ramping capability cannot be deployed in real-time operations. As a solution to the non-delivery issue, we provide a new ramping product designed to improve reliability and reduce the expected operating cost. The trajectory of start-up and shutdown processes is also considered in determining the ramping capability. A new optimization problem is formulated using mixed integer linear programming to be readily applied to practical power system operation. The performance of this proposed method is verified through simulations using a small-scale system and IEEE 118-bus systems. The simulation results demonstrate that the proposed formulation can realize improved generation scheduling alleviating capacity shortages.

Standard economic dispatch problems that consider line losses are linear approximations of a non-convex economic dispatch problem formulated by fixing voltage magnitudes and assuming the decoupling of real and reactive power. This paper formulates and analyzes the general non-convex economic dispatch problem, incorporating and generalizing the Fictitious Nodal Demand (FND) model, resulting in a slack bus independent formulation that provides insight into standard formulations by pointing out commonly used but unnecessary assumptions and by deriving proper choices of "tuning parameters." The proper choice of loss allocation is derived to assign half of the losses of each transmission line to adjacent buses, justifying approaches in the literature. Line constraints are proposed in the form of voltage angle difference limits and are proven equivalent to various other line limits including current magnitude limits and mid-line power flow limits. The formulated general economic dispatch problem with marginal losses consistently models flows and loss approximation, results in approximately correct outcomes and is proven to be reference bus independent. Various approximations of this problem are compared using realistically large transmission network test cases., Comment: 8 pages, 2 figures, 1 table, American Control Conference 2019 (Submitted)

Workplace charging of electric vehicles (EV) from photovoltaic (PV) panels installed on an office building can provide several benefits. This includes the local production and use of PV energy for charging the EV and making use of dynamic tariffs from the grid to schedule the energy exchange with the grid. The long parking time of EV at the workplace provide the chance for the EV to support the grid via vehicle-to-grid technology, the use of a single EV charger for charging several EV by multiplexing and the offer of ancillary services to the grid for up and down regulation. Further, distribution network constraints can be considered to limit the power and prevent the overloading of the grid. A single MILP formulation that considers all the above applications has been proposed in this paper for a charging a fleet of EVs from PV. The MILP is implemented as a receding-horizon model predictive energy management system. Numerical simulation based on market and PV data in Austin, Texas have shown 31% to 650% reduction in the cost of EV charging when compared to immediate and average rate charging policies., Comment: submitted to IEEE Transactions on Smart Grid

In certain electricity markets, because of non-convexities that arise from their operating characteristics, generators that follow the independent system operator's (ISO's) decisions may fail to recover their cost through sales of energy at locational marginal prices. Discriminatory side payments are made by the ISO to incentivize the compliance of generators. Convex hull pricing is a uniform pricing scheme that minimizes these side payments. The Lagrangian dual problem of the unit commitment problem has been used to determine convex hull prices. However, this approach is computationally expensive. In this paper, we propose a polynomially-solvable primal formulation for the Lagrangian dual problem. This formulation explicitly describes for each generating unit the convex hull of its feasible set and the convex envelope of its cost function. We cast our formulation as a second-order cone program when the cost functions are quadratic, and a linear program when the cost functions are piecewise linear. A 96-period 76-unit transmission-constrained example is solved in less than fifteen seconds on a personal computer.

Jamming refers to the deletion, corruption or damage of meter measurements that prevents their further usage. This is distinct from adversarial data injection that changes meter readings while preserving their utility in state estimation. This paper presents a generalized attack regime that uses jamming of secure and insecure measurements to greatly expand the scope of common 'hidden' and 'detectable' data injection attacks in literature. For 'hidden' attacks, it is shown that with jamming, the optimal attack is given by the minimum feasible cut in a specific weighted graph. More importantly, for 'detectable' data attacks, this paper shows that the entire range of relative costs for adversarial jamming and data injection can be divided into three separate regions, with distinct graph-cut based constructions for the optimal attack. Approximate algorithms for attack design are developed and their performances are demonstrated by simulations on IEEE test cases. Further, it is proved that prevention of such attacks require security of all grid measurements. This work comprehensively quantifies the dual adversarial benefits of jamming: (a) reduced attack cost and (b) increased resilience to secure measurements, that strengthen the potency of data attacks., Comment: 11 pages, 8 figures, A version of this will appear in HICSS 2016

Time-varying renewable energy generation can result in serious under-/over-voltage conditions in future distribution grids. Augmenting conventional utility-owned voltage regulating equipment with the reactive power capabilities of distributed generation units is a viable solution. Local control options attaining global voltage regulation optimality at fast convergence rates is the goal here. In this context, novel reactive power control rules are analyzed under a unifying linearized grid model. For single-phase grids, our proximal gradient scheme has computational complexity comparable to that of the rule suggested by the IEEE 1547.8 standard, but it enjoys well-characterized convergence guarantees. Adding memory to the scheme results in accelerated convergence. For three-phase grids, it is shown that reactive injections have a counter-intuitive effect on bus voltage magnitudes across phases. Nevertheless, when our control scheme is applied to unbalanced conditions, it is shown to reach an equilibrium point. Yet this point may not correspond to the minimizer of a voltage regulation problem. Numerical tests using the IEEE 13-bus, the IEEE 123-bus, and a Southern California Edison 47-bus feeder with increased renewable penetration verify the convergence properties of the schemes and their resiliency to grid topology reconfigurations.

Optimized operation of the transmission network is one solution to supply extra demand by more efficient use of transmission facilities, and line switching is one main tool to achieve this goal. In this paper, we add extra constraints to OPF formulation to limit the maximum number of switching operations in every hour based on network conditions, and add switching cost in the objective function to represent extra maintenance cost as a result of frequent switching. We also propose an algorithm to remove less important lines for switching in different loading conditions, so OPF with transmission switching will be solved faster for real-time operation. It is applied to a case study with several operation hours., Comment: 22nd International Symposium on Mathematical Programming, Pittsburgh, July 2015

Meter measurements in the power grid are susceptible to manipulation by adversaries, that can lead to errors in state estimation. This paper presents a general framework to study attacks on state estimation by adversaries capable of injecting bad-data into measurements and further, of jamming their reception. Through these two techniques, a novel `detectable jamming' attack is designed that changes the state estimation despite failing bad-data detection checks. Compared to commonly studied `hidden' data attacks, these attacks have lower costs and a wider feasible operating region. It is shown that the entire domain of jamming costs can be divided into two regions, with distinct graph-cut based formulations for the design of the optimal attack. The most significant insight arising from this result is that the adversarial capability to jam measurements changes the optimal 'detectable jamming' attack design only if the jamming cost is less than half the cost of bad-data injection. A polynomial time approximate algorithm for attack vector construction is developed and its efficacy in attack design is demonstrated through simulations on IEEE test systems., Comment: 8 pages, 7 figures

A coordinated cyber-attack on grid meter readings and breaker statuses can lead to incorrect state estimation that can subsequently destabilize the grid. This paper studies cyber-attacks by an adversary that changes breaker statuses on transmission lines to affect the estimation of the grid topology. The adversary, however, is incapable of changing the value of any meter data and can only block recorded measurements on certain lines from being transmitted to the control center. The proposed framework, with limited resource requirements as compared to standard data attacks, thus extends the scope of cyber-attacks to grids secure from meter corruption. We discuss necessary and sufficient conditions for feasible attacks using a novel graph-coloring based analysis and show that an optimal attack requires breaker status change at only ONE transmission line. The potency of our attack regime is demonstrated through simulations on IEEE test cases., Comment: 5 pages, 5 figures, Accepted to the IEEE PES General Meeting 2015

Data attacks on meter measurements in the power grid can lead to errors in state estimation. This paper presents a new data attack model where an adversary produces changes in state estimation despite failing bad-data detection checks. The adversary achieves its objective by making the estimator incorrectly identify correct measurements as bad data. The proposed attack regime's significance lies in reducing the minimum sizes of successful attacks to more than half of that of undetectable data attacks. Additionally, the attack model is able to construct attacks on systems that are resilient to undetectable attacks. The conditions governing a successful data attack of the proposed model are presented along with guarantees on its performance. The complexity of constructing an optimal attack is discussed and two polynomial time approximate algorithms for attack vector construction are developed. The performance of the proposed algorithms and efficacy of the hidden attack model are demonstrated through simulations on IEEE test systems.

Grid security and open markets are two major smart grid goals. Transparency of market data facilitates a competitive and efficient energy environment, yet it may also reveal critical physical system information. Recovering the grid topology based solely on publicly available market data is explored here. Real-time energy prices are calculated as the Lagrange multipliers of network-constrained economic dispatch; that is, via a linear program (LP) typically solved every 5 minutes. Granted the grid Laplacian is a parameter of this LP, one could infer such a topology-revealing matrix upon observing successive LP dual outcomes. The matrix of spatio-temporal prices is first shown to factor as the product of the inverse Laplacian times a sparse matrix. Leveraging results from sparse matrix decompositions, topology recovery schemes with complementary strengths are subsequently formulated. Solvers scalable to high-dimensional and streaming market data are devised. Numerical validation using real load data on the IEEE 30-bus grid provide useful input for current and future market designs., Comment: Submitted to the IEEE Trans. on Smart Grid

Flexible loads, i.e. the loads whose power trajectory is not bound to a specific one, constitute a sizable portion of current and future electric demand. This flexibility can be used to improve the performance of the grid, should the right incentives be in place. In this paper, we consider the optimal decision making problem faced by a flexible load, demanding a certain amount of energy over its availability period, subject to rate constraints. The load is also capable of providing Ancillary Services (AS) by decreasing or increasing its consumption in response to signals from the Independent System Operator. Under arbitrarily distributed and correlated Markovian energy and AS prices, we obtain the optimal policy for minimizing expected total cost, which includes cost of energy and benefits from AS provision, assuming no capacity reservation requirement for AS provision. We also prove that the optimal policy has a multi-threshold form and can be computed, stored and operated efficiently. We further study the effectiveness of our proposed optimal policy and its impact on the grid. We show that, while optimal simultaneous consumption and AS provision under real-time stochastic prices is achievable with acceptable computational burden, the impact of adopting such real-time pricing schemes on the network might not be as good as suggested by the majority of the existing literature. In fact, we show that such price responsive loads are likely to induce peak-to-average ratios much more than what is observed in the current distribution networks and adversely affect the grid., Comment: submitted to International Journal of Control

Real time operation of the power grid and synchronism of its different elements require accurate estimation of its state variables. Errors in state estimation will lead to sub-optimal Optimal Power Flow (OPF) solutions and subsequent increase in the price of electricity in the market or, potentially overload and create line outages. This paper studies hidden data attacks on power systems by an adversary trying to manipulate state estimators. The adversary gains control of a few meters, and is able to introduce spurious measurements in them. The paper presents a polynomial time algorithm using min-cut calculations to determine the minimum number of measurements an adversary needs to manipulate in order to perform a hidden attack. Greedy techniques are presented to aid the system operator in identifying critical measurements for protection to prevent such hidden data attacks. Secure PMU placement against data attacks is also discussed and an algorithm for placing PMUs for this purpose is developed. The performances of the proposed algorithms are shown through simulations on IEEE test cases.

The potential of recovering the topology of a grid using solely publicly available market data is explored here. In contemporary whole-sale electricity markets, real-time prices are typically determined by solving the network-constrained economic dispatch problem. Under a linear DC model, locational marginal prices (LMPs) correspond to the Lagrange multipliers of the linear program involved. The interesting observation here is that the matrix of spatiotemporally varying LMPs exhibits the following property: Once premultiplied by the weighted grid Laplacian, it yields a low-rank and sparse matrix. Leveraging this rich structure, a regularized maximum likelihood estimator (MLE) is developed to recover the grid Laplacian from the LMPs. The convex optimization problem formulated includes low rank- and sparsity-promoting regularizers, and it is solved using a scalable algorithm. Numerical tests on prices generated for the IEEE 14-bus benchmark provide encouraging topology recovery results., Comment: PES General Meeting 2014 submission

In this manuscript we continue the thread of [M. Chertkov, F. Pan, M. Stepanov, Predicting Failures in Power Grids: The Case of Static Overloads, IEEE Smart Grid 2011] and suggest a new algorithm discovering most probable extreme stochastic events in static power grids associated with intermittent generation of wind turbines. The algorithm becomes EXACT and EFFICIENT (polynomial) in the case of the proportional (or other low parametric) control of standard generation, and log-concave probability distribution of the renewable generation, assumed known from the wind forecast. We illustrate the algorithm's ability to discover problematic extreme events on the example of the IEEE RTS-96 model of transmission with additions of 10%, 20% and 30% of renewable generation. We observe that the probability of failure may grow but it may also decrease with increase in renewable penetration, if the latter is sufficiently diversified and distributed., Comment: 7 pages, 3 figures, invited session on Smart Grid Integration of Renewable Energy: Failure analysis, Microgrids, and Estimation at CDC/ECC 2011