Search

This two-part tutorial dives into the field of data-driven power flow linearization (DPFL), a domain gaining increased attention. DPFL stands out for its higher approximation accuracy, wide adaptability, and better ability to implicitly incorporate the latest system attributes. This renders DPFL a potentially superior option for managing the significant fluctuations from renewable energy sources, a step towards realizing a more sustainable energy future, by translating the higher model accuracy into increased economic efficiency and less energy losses. To conduct a deep and rigorous reexamination, this tutorial first classifies existing DPFL methods into DPFL training algorithms and supportive techniques. Their mathematical models, analytical solutions, capabilities, limitations, and generalizability are systematically examined, discussed, and summarized. In addition, this tutorial reviews existing DPFL experiments, examining the settings of test systems, the fidelity of datasets, and the comparison made among a limited number of DPFL methods. Further, this tutorial implements extensive numerical comparisons of all existing DPFL methods (40 methods in total) and four classic physics-driven approaches, focusing on their generalizability, applicability, accuracy, and computational efficiency. Through these simulationmethodss, this tutorial aims to reveal the actual performance of all the methods (including the performances exposed to data noise or outliers), guiding the selection of appropriate linearization methods. Furthermore, this tutorial discusses future directions based on the theoretical and numerical insights gained. As the first part, this paper reexamines DPFL theories, covering all the training algorithms and supportive techniques. Capabilities, limitations, and aspects of generalizability, which were previously unmentioned in the literature, have been identified., Comment: 20 pages

Building on the theoretical insights of Part I, this paper, as the second part of the tutorial, dives deeper into data-driven power flow linearization (DPFL), focusing on comprehensive numerical testing. The necessity of these simulations stems from the theoretical analysis's inherent limitations, particularly the challenge of identifying the differences in real-world performance among DPFL methods with overlapping theoretical capabilities and/or limitations. The absence of a comprehensive numerical comparison of DPFL approaches in the literature also motivates this paper, especially given the fact that over 95% of existing DPFL studies have not provided any open-source codes. To bridge the gap, this paper first reviews existing DPFL experiments, examining the adopted test scenarios, load fluctuation settings, data sources, considerations for data noise/outliers, and the comparison made so far. Subsequently, this paper evaluates a total of 44 methods, containing over 30 existing DPFL approaches, some innovative DPFL techniques, and several classic physics-driven power flow linearization methods for benchmarking. The evaluation spans various dimensions, including generalizability, applicability, accuracy, and computational efficiency, using various different test cases scaling from 9-bus to 1354-bus systems. The numerical analysis identifies and examines significant trends and consistent findings across all methods under various test cases. It also offers theoretical insights into phenomena like under-performance, failure, excessive computation times, etc. Overall, this paper identifies the differences in the performances of the wide range of DPFL methods, reveals gaps not evident from theoretical discussions, assists in method selection for real-world applications, and provides thorough discussions on open questions within DPFL research, indicating ten potential future directions., Comment: 26 pages

The growing renewable energy sources have posed significant challenges to traditional power scheduling. It is difficult for operators to obtain accurate day-ahead forecasts of renewable generation, thereby requiring the future scheduling system to make real-time scheduling decisions aligning with ultra-short-term forecasts. Restricted by the computation speed, traditional optimization-based methods can not solve this problem. Recent developments in reinforcement learning (RL) have demonstrated the potential to solve this challenge. However, the existing RL methods are inadequate in terms of constraint complexity, algorithm performance, and environment fidelity. We are the first to propose a systematic solution based on the state-of-the-art reinforcement learning algorithm and the real power grid environment. The proposed approach enables planning and finer time resolution adjustments of power generators, including unit commitment and economic dispatch, thus increasing the grid's ability to admit more renewable energy. The well-trained scheduling agent significantly reduces renewable curtailment and load shedding, which are issues arising from traditional scheduling's reliance on inaccurate day-ahead forecasts. High-frequency control decisions exploit the existing units' flexibility, reducing the power grid's dependence on hardware transformations and saving investment and operating costs, as demonstrated in experimental results. This research exhibits the potential of reinforcement learning in promoting low-carbon and intelligent power systems and represents a solid step toward sustainable electricity generation., Comment: 12 pages, 7 figures

The superior performance of deep learning relies heavily on a large collection of sample data, but the data insufficiency problem turns out to be relatively common in global electricity markets. How to prevent overfitting in this case becomes a fundamental challenge when training deep learning models in different market applications. With this in mind, we propose a general framework, namely Knowledge-Augmented Training (KAT), to improve the sample efficiency, and the main idea is to incorporate domain knowledge into the training procedures of deep learning models. Specifically, we propose a novel data augmentation technique to generate some synthetic data, which are later processed by an improved training strategy. This KAT methodology follows and realizes the idea of combining analytical and deep learning models together. Modern learning theories demonstrate the effectiveness of our method in terms of effective prediction error feedbacks, a reliable loss function, and rich gradient noises. At last, we study two popular applications in detail: user modeling and probabilistic price forecasting. The proposed method outperforms other competitors in all numerical tests, and the underlying reasons are explained by further statistical and visualization results., Comment: Accepted by IEEE Transactions on Power Systems, 12 pages, 11 figures, 6 tables

Intervention policies against COVID-19 have caused large-scale disruptions globally, and led to a series of pattern changes in the power system operation. Analyzing these pandemic-induced patterns is imperative to identify the potential risks and impacts of this extreme event. With this purpose, we developed an open-access data hub (COVID-EMDA+), an open-source toolbox (CoVEMDA), and a few evaluation methods to explore what the U.S. power systems are experiencing during COVID-19. These resources could be broadly used for research, public policy, and educational purposes. Technically, our data hub harmonizes a variety of raw data such as generation mix, demand profiles, electricity price, weather observations, mobility, confirmed cases and deaths. Typical methods are reformulated and standardized in our toolbox, including baseline estimation, regression analysis, and scientific visualization. Here the fluctuation index and probabilistic baseline are proposed for the first time to consider data fluctuation and estimation uncertainty. Based on these, we conduct three empirical studies on the U.S. power systems, and share new solutions and unexpected findings to address the issues of public concerns. This conveys a more complete picture of the pandemic's impacts, and also opens up several attractive topics for future work. Python, Matlab source codes, and user manuals are all publicly shared on a Github repository., Comment: Journal accepted by IEEE Trans on Power Systems, 12 pages, 7 figures, 5 tables. Website: https://github.com/tamu-engineering-research/COVID-EMDA

There is an opportunity in modern power systems to explore the demand flexibility by incentivizing consumers with dynamic prices. In this paper, we quantify demand flexibility using an efficient tool called time-varying elasticity, whose value may change depending on the prices and decision dynamics. This tool is particularly useful for evaluating the demand response potential and system reliability. Recent empirical evidences have highlighted some abnormal features when studying demand flexibility, such as delayed responses and vanishing elasticities after price spikes. Existing methods fail to capture these complicated features because they heavily rely on some predefined (often over-simplified) regression expressions. Instead, this paper proposes a model-free methodology to automatically and accurately derive the optimal estimation pattern. We further develop a two-stage estimation process with Siamese long short-term memory (LSTM) networks. Here, a LSTM network encodes the price response, while the other network estimates the time-varying elasticities. In the case study, the proposed framework and models are validated to achieve higher overall estimation accuracy and better description for various abnormal features when compared with the state-of-the-art methods., Comment: Author copy of the manuscript submitted to IEEE Trans on Power Systems

Distribution grid topology and admittance information are essential for system planning, operation, and protection. In many distribution grids, missing or inaccurate topology and admittance data call for efficient estimation methods. However, measurement data may be insufficient or contaminated with large noise, which will introduce fundamental limits to the estimation accuracy. This work explores the theoretical precision limits of the topology and admittance estimation (TAE) problem, with different measurement devices, noise levels, and the number of measurements. On this basis, we propose a conservative progressive self-adaptive (CPS) algorithm to estimate the topology and admittance. Results on IEEE 33 and 141-bus systems validate that the proposed CPS method can approach the theoretical precision limits under various measurement settings.

The novel coronavirus disease (COVID-19) has rapidly spread around the globe in 2020, with the U.S. becoming the epicenter of COVID-19 cases since late March. As the U.S. begins to gradually resume economic activity, it is imperative for policymakers and power system operators to take a scientific approach to understanding and predicting the impact on the electricity sector. Here, we release a first-of-its-kind cross-domain open-access data hub, integrating data from across all existing U.S. wholesale electricity markets with COVID-19 case, weather, cellular location, and satellite imaging data. Leveraging cross-domain insights from public health and mobility data, we uncover a significant reduction in electricity consumption across that is strongly correlated with the rise in the number of COVID-19 cases, degree of social distancing, and level of commercial activity., Comment: This paper has been accepted for publication by Joule. The manuscript can also be accessed from EnerarXiv: http://www.enerarxiv.org/page/thesis.html?id=1989

The considerable cessation of human activities during the COVID-19 pandemic has affected global energy use and CO2 emissions. Here we show the unprecedented decrease in global fossil CO2 emissions from January to April 2020 was of 7.8% (938 Mt CO2 with a +6.8% of 2-{\sigma} uncertainty) when compared with the period last year. In addition other emerging estimates of COVID impacts based on monthly energy supply or estimated parameters, this study contributes to another step that constructed the near-real-time daily CO2 emission inventories based on activity from power generation (for 29 countries), industry (for 73 countries), road transportation (for 406 cities), aviation and maritime transportation and commercial and residential sectors emissions (for 206 countries). The estimates distinguished the decline of CO2 due to COVID-19 from the daily, weekly and seasonal variations as well as the holiday events. The COVID-related decreases in CO2 emissions in road transportation (340.4 Mt CO2, -15.5%), power (292.5 Mt CO2, -6.4% compared to 2019), industry (136.2 Mt CO2, -4.4%), aviation (92.8 Mt CO2, -28.9%), residential (43.4 Mt CO2, -2.7%), and international shipping (35.9Mt CO2, -15%). Regionally, decreases in China were the largest and earliest (234.5 Mt CO2,-6.9%), followed by Europe (EU-27 & UK) (138.3 Mt CO2, -12.0%) and the U.S. (162.4 Mt CO2, -9.5%). The declines of CO2 are consistent with regional nitrogen oxides concentrations observed by satellites and ground-based networks, but the calculated signal of emissions decreases (about 1Gt CO2) will have little impacts (less than 0.13ppm by April 30, 2020) on the overserved global CO2 concertation. However, with observed fast CO2 recovery in China and partial re-opening globally, our findings suggest the longer-term effects on CO2 emissions are unknown and should be carefully monitored using multiple measures.

Increasing the penetration of variable generation has a substantial effect on the operational reliability of power systems. The higher level of uncertainty that stems from this variability makes it more difficult to determine whether a given operating condition will be secure or insecure. Data-driven techniques provide a promising way to identify security rules that can be embedded in economic dispatch model to keep power system operating states secure. This paper proposes using a sparse weighted oblique decision tree to learn accurate, understandable, and embeddable security rules that are linear and can be extracted as sparse matrices using a recursive algorithm. These matrices can then be easily embedded as security constraints in power system economic dispatch calculations using the Big-M method. Tests on several large datasets with high renewable energy penetration demonstrate the effectiveness of the proposed method. In particular, the sparse weighted oblique decision tree outperforms the state-of-art weighted oblique decision tree while keeping the security rules simple. When embedded in the economic dispatch, these rules significantly increase the percentage of secure states and reduce the average solution time., Comment: 8 pages, 6 figures

Power system cascading failures become more time variant and complex because of the increasing network interconnection and higher renewable energy penetration. High computational cost is the main obstacle for a more frequent online cascading failure search, which is essential to improve system security. In this work, we show that the complex mechanism of cascading failures can be well captured by training a graph convolutional network (GCN) offline. Subsequently, the search of cascading failures can be significantly accelerated with the aid of the trained GCN model. We link the power network topology with the structure of the GCN, yielding a smaller parameter space to learn the complex mechanism. We further enable the interpretability of the GCN model by a layer-wise relevance propagation (LRP) algorithm. The proposed method is tested on both the IEEE RTS-79 test system and China's Henan Province power system. The results show that the GCN guided method can not only accelerate the search of cascading failures, but also reveal the reasons for predicting the potential cascading failures., Comment: 9 pages,8 figures

Cascading failures have become a severe threat to interconnected modern power systems. The ultrahigh complexity of the interconnected networks is the main challenge toward the understanding and management of cascading failures. In addition, high penetration of wind power integration introduces large uncertainties and further complicates the problem into a massive scenario simulation problem. This paper proposes a framework that enables a fast cascading path searching under high penetration of wind power. In addition, we ease the computational burden by formulating the cascading path searching problem into a Markov chain searching problem and further use a dictionary-based technique to accelerate the calculations. In detail, we first generate massive wind generation and load scenarios. Then, we utilize the Markov search strategy to decouple the problem into a large number of DC power flow (DCPF) and DC optimal power flow (DCOPF) problems. The major time-consuming part, the DCOPF and the DCPF problems, is accelerated by the dynamic construction of a line status dictionary (LSD). The information in the LSD can significantly ease the computation burden of the following DCPF and DCOPF problems. The proposed method is proven to be effective by a case study of the IEEE RTS-79 test system and an empirical study of China's Henan Province power system., Comment: 10 pages, 5 figures, accepted by IEEE Transactions on Sustainable Energy

Data-driven models analyze power grids under incomplete physical information, and their accuracy has been mostly validated empirically using certain training and testing datasets. This paper explores error bounds for data-driven models under all possible training and testing scenarios, and proposes an evaluation implementation based on Rademacher complexity theory. We answer key questions for data-driven models: how much training data is required to guarantee a certain error bound, and how partial physical knowledge can be utilized to reduce the required amount of data. Our results are crucial for the evaluation and application of data-driven models in power grid analysis. We demonstrate the proposed method by finding generalization error bounds for two applications, i.e. branch flow linearization and external network equivalent under different degrees of physical knowledge. Results identify how the bounds decrease with additional power grid physical knowledge or more training data., Comment: 11 pages, 10 fugures

Lithium-ion battery systems have been used in practical power systems for peak-shaving, demand response, and frequency regulation. However, a lithium-ion battery is degrading while cycling and would be scrapped when the capacity reduces to a certain threshold (e.g. 80%). Such scrapping criterion may not explore the maximum benefit from the battery storage. In this paper, we propose a novel scrapping criterion for peak-shaving energy storage based on battery efficiency, time-of-use price, and arbitrage benefit. A new battery life model with scrapping parameters is then derived using this criterion. Embedded with the life model, an optimal operation method for peak-shaving energy storage system is presented. The results of case study show that the operation method could maximize the benefits of peak-shaving energy storage while delaying battery degradation. Compared with the traditional 80% capacity-based scrapping criterion, our efficiency-based scrapping criterion can significantly improve the lifetime benefit of the battery., Comment: Add references

The high penetration of photovoltaic (PV) is reshaping the electricity net-load curve and has a significant impact on power system operation and planning. The concept of duck curve is widely used to describe the timing imbalance between peak demand and PV generation. The traditional duck curve is deterministic and only shows a single extreme or typical scenario during a day. Thus, it cannot capture both the probability of that scenario and the uncertainty of PV generation and loads. These weaknesses limit the application of the duck curve on power system planning under high PV penetration. To address this issue, the novel concepts of probabilistic duck curve (PDC) and probabilistic ramp curve (PRC) are proposed to accurately model the uncertainty and variability of electricity net load and ramp under high PV penetration. An efficient method is presented for modeling PDC and PRC using kernel density estimation, copula function, and dependent discrete convolution. Several indices are designed to quantify the characteristics of the PDC and PRC. For the application, we demonstrate how the PDC and PRC will benefit flexible resource planning. Finally, an empirical study on the Qinghai provincial power system of China validates the effectiveness of the presented method. The results of PDC and PRC intuitively illustrate that the ramp demand and the valley of net load face considerable uncertainty under high PV penetration. The results of flexible resource planning indicate that retrofitting coal-fired units has remarkable performance on enhancing the power system flexibility in Qinghai. In average, reducing the minimal output of coal-fired units by 1 MW will increase PV accommodation by over 4 MWh each day.

The modeling of multi-energy systems (MES) is the basic task of analyzing energy systems integration. The variable energy efficiencies of the energy conversion and storage components in MES introduce nonlinearity to the model and thus complicate the analysis and optimization of MES. In this paper, we propose a standardized matrix modeling approach to automatically model MES with variable energy efficiencies based on the energy hub (EH) modeling framework. We use piecewise linearization to approximate the variable energy efficiencies; as a result, a component with variable efficiency is equivalent to several parallel components with constant efficiencies. The nonlinear energy conversion and storage relationship in EH can thus be further modeled under a linear modeling framework using matrices. Such matrix modeling approach makes the modeling of an arbitrary EH with nonlinear energy components highly automated by computers. The proposed modeling approach can further facilitate the operation and planning optimization of EH with variable efficiencies. Case studies are presented to show how the nonlinear approximation accuracy and calculation efficiency can be balanced using the proposed model in the optimal operation of EH., Comment: 10 pages, 12 figures

Multi-energy systems can provide a constant level of service to end-use energy demands, while deriving delivered energy from a variety of primary/secondary energy sources. This fuel-switching capability can be used to reduce operating expenses, reduce environmental impacts, improve flexibility to accommodate renewable energy, and improve reliability. This paper presents four frameworks for incentivizing energy hub equipment investments for low-carbon operation targets. These frameworks vary in the measures taken to achieve low-carbon operation (explicit constraint vs. carbon pricing) and in the relationship between the hub builder and operator (cooperative vs. uncoordinated). The underlying energy hub model upon which these frameworks are built is an enhanced greenfield model, introducing 'energy buses' to reduce dimensionality. A case study is conducted for a campus being designed in Beijing, and results from each framework are compared to illustrate their relative costs. When the operator cannot be trusted to cooperate in controlling emissions, the system must be 'overbuilt' with more expensive equipment to ensure emissions target are met. A taxation-based approach increases overall costs at moderate emissions targets, but this effect decreases at aggressive targets. This paper also compares the cost of less efficient institutional frameworks with the most efficient approach, i.e. cooperation between builder and operator with constraints on emissions.

Probabilistic load forecasts provide comprehensive information about future load uncertainties. In recent years, many methodologies and techniques have been proposed for probabilistic load forecasting. Forecast combination, a widely recognized best practice in point forecasting literature, has never been formally adopted to combine probabilistic load forecasts. This paper proposes a constrained quantile regression averaging (CQRA) method to create an improved ensemble from several individual probabilistic forecasts. We formulate the CQRA parameter estimation problem as a linear program with the objective of minimizing the pinball loss, with the constraints that the parameters are nonnegative and summing up to one. We demonstrate the effectiveness of the proposed method using two publicly available datasets, the ISO New England data and Irish smart meter data. Comparing with the best individual probabilistic forecast, the ensemble can reduce the pinball score by 4.39% on average. The proposed ensemble also demonstrates superior performance over nine other benchmark ensembles., Comment: Submitted to IEEE Transactions on Smart Grid

The widespread popularity of smart meters enables an immense amount of fine-grained electricity consumption data to be collected. Meanwhile, the deregulation of the power industry, particularly on the delivery side, has continuously been moving forward worldwide. How to employ massive smart meter data to promote and enhance the efficiency and sustainability of the power grid is a pressing issue. To date, substantial works have been conducted on smart meter data analytics. To provide a comprehensive overview of the current research and to identify challenges for future research, this paper conducts an application-oriented review of smart meter data analytics. Following the three stages of analytics, namely, descriptive, predictive and prescriptive analytics, we identify the key application areas as load analysis, load forecasting, and load management. We also review the techniques and methodologies adopted or developed to address each application. In addition, we also discuss some research trends, such as big data issues, novel machine learning technologies, new business models, the transition of energy systems, and data privacy and security., Comment: IEEE Transactions on Smart Grid, 2018

The linearization of a power flow (PF) model is an important approach for simplifying and accelerating the calculation of a power system's control, operation, and optimization. Traditional model-based methods derive linearized PF models by making approximations in the analytical PF model according to the physical characteristics of the power system. Today, more measurements of the power system are available and thus facilitate data-driven approaches beyond model-driven approaches. This work studies a linearized PF model through a data-driven approach. Both a forward regression model ((P, Q) as a function of (theta, V)) and an inverse regression model ((theta, V) as a function of (P, Q)) are proposed. Partial least square (PLS)- and Bayesian linear regression (BLR)-based algorithms are designed to address data collinearity and avoid overfitting. The proposed approach is tested on a series of IEEE standard cases, which include both meshed transmission grids and radial distribution grids, with both Monte Carlo simulated data and public testing data. The results show that the proposed approach can realize a higher calculation accuracy than model-based approaches can. The results also demonstrate that the obtained regression parameter matrices of data-driven models reflect power system physics by demonstrating similar patterns with some power system matrices (e.g., the admittance matrix)., Comment: 9 pages, 7 figures, 1 table