Search

This note shows an elegant relationship between the quadratic optimal control and robust stabilization for linear time-invariant (LTI) systems, where the former control can robustly stabilize the latter system, provided that the matched uncertainty is bounded. Through reviewing the relevant literature, some common mistakes in regard to this relationship are found. The correct results are obtained and proved in both frequency and time domains. The results are applicable to both single- and multi-input cases. They are significant as the simple LQR design for the nominal system can be utilized to directly solve—with no further effort—the complex robust stabilization problem for a class of linear uncertain systems. [ABSTRACT FROM AUTHOR]

Ensemble numerical weather prediction (NWP) is the backbone of the state-of-the-art solar forecasting for horizons ranging between a few hours and a few days. Dynamical ensemble forecasts are generated by perturbing the initial condition, and thereby obtaining a set of equally likely trajectories of the future weather. Generating dynamical ensemble forecasts demands extensive knowledge of atmospheric science and significant computational resources. Hence, the task is often performed by international and national weather centers and space agencies. Solar forecasters, on the other hand, are primarily interested in post-processing those ensemble forecasts disseminated by weather service providers, as to arrive at forecasts of solar power output. To facilitate the uptake of ensemble NWP forecasts in solar power forecasting research, this paper offers an archived dataset from the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System, over a four-year period (2017–2020) and over an extensive geographical region (e.g., most of Europe and North America), under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Two case studies are presented to demonstrate the usage of the dataset. One case study elaborates how ensemble forecasts can be summarized and calibrated, which constitute two common forms of probabilistic forecast post-processing. The other demonstrates how the dataset can be used in solar power forecasting applications, which compares machine learning with the physical model chain in terms of their irradiance-to-power conversion capability. The Python code used to produce the results shown in this paper is made available on GitHub. • Four years of ECMWF EPS forecasts are offered under CC BY 4.0 license. • Dynamical ensemble GHI forecasts are provided over US and Europe. • The dataset facilitates a range of solar forecasting applications. • Two case studies on post-processing and model chain construction are included. [ABSTRACT FROM AUTHOR]

Recent literature has confirmed the benefits of jointly optimizing and allocating various firm power enablers, such as photovoltaic (PV) overbuilding & proactive curtailment, geographical smoothing, or energy storage. These enablers facilitate the transformation of variable PV power into effectively dispatchable power, thereby firming up PV generation. However, many previous studies on firm PV generation only considered batteries as the energy storage option, which notoriously elevates the overall system costs owing to the short-duration nature of battery storage. Besides, the implications of the anticipated yet uncertain decrease in storage costs on the economic viability of firm PV and system component sizes remain unclear. This work, therefore, introduces hydrogen as a long-duration (e.g. , seasonal) storage option and elucidates the differences between short- and long-duration storage in reducing the cost of firm PV power. Specifically, two facets separate this work from its antecedents: (1) A mixed-integer linear programming model that minimizes the firm kWh premium of the PV–battery–hydrogen system, which possesses the role of both short- and long-duration storage, is proposed to determine the optimal system configuration; (2) the impact of changes in storage options and costs on the energy component ratings is investigated, and the scaling of system economics with these changes is assessed. The analysis reveals that the obtained firm kWh premium stands at 5.42 when the firm 100% PV-supplied system is utilized to fulfill the load demand with an average daily value of 22.04 MWh, while the installation of a 44.81-MWh battery, a 684-kW electrolyzer, and a 540-kW fuel cell, is required to achieve the optimal system costs. Additionally, compared to the future cost change in long-duration storage due to technology updates, the premium is more sensitive to an equal amount of change in the cost of short-duration storage. The results can offer policymakers actionable insights regarding the capacity optimization of PV plants, the strategic deployment of hydrogen systems, and the cost-effective construction of zero-carbon energy networks. [Display omitted] • A model is proposed to optimize the cost of firm PV generation. • The battery, a short-duration storage option, is mainly employed for diurnal storage. • The hydrogen system (long-duration storage) primarily caters to inter-seasonal storage. • The incorporation of long-duration storage lowers the system premium by 10%. • Battery cost reduction diminishes the system cost more than the hydrogen system. [ABSTRACT FROM AUTHOR]

Is the post-processing of global horizontal irradiance (GHI) forecasts necessary for issuing good photovoltaic (PV) power forecasts? Whenever this question is raised, the instinctive supposition always seems to be "yes," because GHI is the most important weather parameter governing the amount of PV power generated, and surely, the better the GHI forecasts are, the better the PV power forecasts should result. To attend to this question more scientifically and more formally, two classic deterministic-to-deterministic post-processing methods, namely, the model output statistics and kernel conditional density estimation, are applied at various stages of PV power forecasting, resulting in four distinct workflows. These different workflows are trained and tested on three PV plants in Hungary, using data from a four-year (2017–2020) period. Both ground-based GHI and satellite-retrieved GHI are used as the "truth" with which numerical weather prediction (NWP) GHI forecasts are post-processed. A very thorough deterministic forecast verification exercise is conducted following the best practices. It is found that contrary to the common supposition, post-processing GHI only leads to marginal, if that can be quantified at all, benefits, so long as the PV power forecasts are to be post-processed. This ought to be deemed as a very important finding, as it puts into question the "GHI forecasting + post-processing + irradiance-to-power conversion" workflow that has dominated solar forecasting for decades. • Post-processing is best performed on PV power instead of GHI • Model chains change the bias and variance of the input forecasts • Post-processing methods developed for GHI are transferable to PV power [ABSTRACT FROM AUTHOR]

Numerical weather prediction (NWP) has hitherto been the default tool for providing day-ahead forecasting services to the solar energy industry. Rapid advancements in solar forecasting using NWP call for more appropriate forecast verification procedures. Current solar forecast verification is almost always carried out through ground-based radiometric data collected at point locations. Consequently, spatial features embedded in the gridded NWP forecasts cannot be verified. This study presents the spatial verification of solar irradiance forecasts using the neighborhood method, with the main goal of emphasizing the importance of such verification procedures. By applying spatial smoothing one establishes a way to directly compare the observed and forecast fields, and concurrently, mitigate verification errors that may arise from small-scale spatial displacements. Within this framework, two variants of the neighborhood-based verification, namely, the fraction-field method and the upscaling method, are examined with respect to two reanalysis products, namely, ERA5 and MERRA-2. The results suggest that, in comparison to the upscaling method, the fraction-field method can better quantify forecast performance by providing fractions skill scores. On top of the traditional neighborhood approach, which involves the subjective selection of threshold for dichotomization, an automatic threshold segmentation method based on the three-component skew-normal mixture model is proposed to resolve the issue, which can also lead to substantial time savings in data processing. Given the spatio-temporal attributes and benefits of visualization, spatial verification is anticipated to serve as a complementary practice to the current mainstream point-location forecast verification. [Display omitted] • The spatial solar forecast verification with neighborhood method is developed. • Automatic threshold segmentation with skew-normal mixture is proposed. • The forecast performance is quantified via explicit fractions skill scores. • ERA5 presents superior forecasting performance compared to MERRA-2. [ABSTRACT FROM AUTHOR]

Solar resource assessments most generally require atmospheric information, which is customarily acquired from gridded datasets. The spatial scale mismatch problem, i.e., the difference in spatial representativeness of gridded data and in situ measurements, therefore becomes relevant. This study examines how the gridded data used as inputs to clear-sky radiation models can affect their performance at urban scale. The tropical island of Singapore is selected for the case study. Aerosol optical depth at 550 nm (AOD550), Å ngström exponent (AE), and precipitable water (PW) from both the MERRA-2 reanalysis and ground-based stations (AERONET and SuomiNet) are collected between 2013–2020. Firstly, it is found that, relatively to the AERONET ground truth, the bias in MERRA-2's AOD550 is more prominent than that in AE or PW. Next, the bias propagation from the gridded inputs (AOD550, AE, and PW) to clear-sky radiation predictions is explored using various models. The estimated clear-sky direct normal irradiance (DNIcs) is more sensitive to AOD550 variation than the clear-sky global horizontal irradiance (GHIcs). Six clear-sky radiation models, five of which accept MERRA-2 gridded inputs, are compared with each other, and with the in situ irradiance measurements recorded at 9 sites. The inter-model difference across Singapore is remarkably consistent because the whole island fits inside a single MERRA-2 grid cell. Under high-AOD550 situations, however, the inter-model deviation becomes large for both GHIcs and DNIcs. The conventional model-versus-measurement comparison shows that each model achieves very different site-to-site performance, largely because the spatially-averaged inputs cannot fully represent the micro-climatic variability. Relatively speaking, no clear-sky radiation model significantly outperforms its peers. The simple MAC2 model and the empirical (locally derived) Yang GHIcs-only model are recommended for Singapore. • MERRA-2 reanalysis aerosol optical depth and water vapor validation using two AERONET stations at Singapore. • Aerosol and water vapor biases impact clear-sky irradiance modeling. • Clear-sky radiation models evaluated in situ and intercompared at 9 sites. • Risks and benefits of using MERRA-2 atmospheric data over small areas. [ABSTRACT FROM AUTHOR]

Addressing the challenges of achieving United Nations Sustainable Development Goal SDG2 is complex, especially considering the profound impacts of population growth, environmental degradation, and disruptions from the COVID-19 pandemic and armed conflicts on the global food supply system. Unjustifiable farmland abandonment poses a critical obstacle to food security, demanding a comprehensive knowledge framework for sensible decision-making in abandoned farmland reclamation. However, current research lacks systematic solutions. This research seeks to address the challenges of farmland abandonment through a comprehensive knowledge framework. Integrating remote sensing surveillance, mechanism analysis, and scenario projection, the primary objective is to develop the abandoned farmland simulation model (AFSM) by combining abandoned farmland identification, the system dynamics (SD), and CLUE-S. The AFSM facilitates explicit spatiotemporal abandonment simulation, contributing to a deeper understanding of the dynamic evolution and mechanisms involved in farmland abandonment. The AFSM is formulated through a logical sequence of "spatiotemporal data acquisition – quantity prognostication – spatial simulation." The process begins by tracking annual land use changes, identifying spatio-temporal changes in abandoned farmland. Subsequently, the research employs the SD model to establish an elucidative framework for abandonment mechanisms, facilitating the quantitative analysis of factors influencing farmers' decisions regarding farmland abandonment. Finally, the CLUE-S model is utilized to prognosticate the spatial abandonment trend. All modules of the model have passed the precision test and inspection. The AFSM outcomes reveal that the abandonment rate in the city of Jingdezhen fluctuated between 5.03% and 13.21% from 2003 to 2020, averaging 8.37%. The model systematically quantifies the multiple socio-economic variables that impact abandonment decisions. Key parameter groups delineate distinct development scenarios and make predictions. By 2032, the abandonment rate under the scenario of farmland protection is projected to be 10.88%, markedly lower than the inertial development and economic priority scenarios by 15.24% and 13.43%, respectively. The concentration of abandoned farmland is primarily observed in elevated, steep-sloped areas, exhibiting a "climbing mountain and slope" trend. The AFSM emerges as a crucial tool for policymakers, farmland managers, and researchers, empowering them to make well-informed decisions concerning abandoned farmland reclamation. The explicit spatio-temporal simulation, complemented by a clarifying framework for abandonment mechanisms, enables the quantification of the influence of various factors on farmland abandonment decision-making and realizes the coupling of quantitative and spatial simulation under various scenarios. [Display omitted] • Global unjustifiable abandonment threatens food security, necessitating a systematic knowledge framework and solutions. • The abandoned farmland simulation model (AFSM) achieves the coupling of quantitative and spatial simulations. • The quantitative simulation accuracy ranges from 89% to 92%, while the spatial simulation accuracy is 80.75%. • The farmland protection scenario is most conducive to curbing abandonment, but it limits the income growth rate of farmers. • Enhancing the cost-effectiveness of farming and developing new types of agricultural entities can curb abandonment. [ABSTRACT FROM AUTHOR]

Global horizontal irradiance (GHI) forecasting at intra-day horizons of up to 12-h ahead is vital to grid integration of solar photovoltaics, but has been fundamentally difficult for all methods that do not involve numerical weather prediction (NWP), since non-NWP methods are unable to extrapolate the data to a horizon beyond a fews hours. The European Centre for Medium-Range Weather Forecasts (ECMWF) and the National Oceanic and Atmospheric Administration (NOAA) are the most representative weather centers in Europe and America, respectively. To understand their operational impact and value to grid integration, the ECMWF's High Resolution (HRES) model and two models from NOAA, namely, Rapid Refresh (RAP) and High-Resolution Rapid Refresh (HRRR), are validated through the Murphy–Winkler distribution-oriented verification framework, over the year 2020, at seven locations. All forecasts are retrieved at native horizontal resolutions—9 km for HRES, 13 km for RAP, and 3 km for HRRR; it depicts the "off-the-shelf" scenario if these forecasts are to be utilized by end users. Three simple linear correction methods, each being statistically optimal in its own respect, are used to post-process the raw forecasts. It was found that 1–12-h-ahead ECMWF's HRES forecasts have a significantly lower root mean square error (14.0–33.7%) as compared to NOAA's HRRR (19.0–53.2%) and RAP (19.2–45.9%). Even after the large biases in HRRR and RAP forecasts are removed, those post-processed versions are still inferior to the raw HRES forecasts. • Twelve-hour-ahead operational solar forecasts from ECMWF and NOAA are verified. • Murphy–Winkler distribution forecast verification framework is used. • Three optimal linear post-processing schemes are proposed and discussed. • ECMWF forecasts are found to be of higher quality than the NOAA forecasts. • The normalized RMSE of ECMWF's High Resolution forecasts ranges from 14% to 34%. [ABSTRACT FROM AUTHOR]

Weather is often found to be a key driving factor for power generation and energy consumption. To capture the effects of weather, many energy forecasting practices, such as load forecasting, renewable power generation forecasting, gas and electricity price forecasting, and power distribution systems outage forecasting, would rely on numerical weather prediction (NWP). In the academic literature, however, energy forecasting models have often been developed based on settings of ex post forecasting, where the actual observations of weather variables during the forecasted period are being used. Such gap between academic research and field practices is partly due to the shortage of historical weather forecasts. To that end, an NWP forecast dataset from the European Centre for Medium-Range Weather Forecasts (ECMWF) High Resolution (HRES) model, as available in the ECMWF's Archive Catalogue, is offered to the energy forecasting community under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Since the raw data is massive in size, a subset which is thought sufficient for energy forecasting research purposes is provided through this article. Four years (2017–2020) of HRES forecasts of 14 frequently used weather variables, over the geographical region bounded by 63° N, − 126 °W, 21° S, and 36° E (most of Europe and North America), on a 0. 5 °by 0. 5 °longitude/latitude grid, are released in the form of NetCDF files. This dataset is able to support a variety of aforementioned energy forecasting tasks. In addition to introducing various means to utilize the dataset, this article provides a set of case studies on post-processing of day-ahead solar forecasts. The R code being used to produce the results shown in this article is also made available, so that the readers can reproduce this case study as well as adopt the code for other relevant studies. • Four years of operational forecasts from the ECMWF HRES model are offered. • The dataset allows energy forecasting applications such as solar, wind, or load forecasting. • ECMWF operational model constitutes the world's best global NWP model to date. • Case studies on solar forecast post-processing are included to exemplify the usage of the dataset. [ABSTRACT FROM AUTHOR]

Site adaptation refers to procedures for correcting systematic errors in an extended period of gridded modeled data using a short period of ground-based measurements used as unbiased reference. Traditionally, site adaptation leverages a single gridded product and issues point predictions. Currently, remote-sensed and reanalysis data are available from different sources providing multiple versions of estimates of a same atmospheric variable, for any location on Earth. These datasets allow what is called an ensemble prediction. In this regard, this contribution proposes a probabilistic site-adaption framework, and describes how one can use parametric and nonparametric techniques within the framework. On top of the stand-alone probabilistic site-adaption methods, heuristics are optionally used to combine quantiles, to further improve the accuracy of site adaptation. To exemplify the framework, global horizontal irradiance data from 26 sites worldwide with different climate characteristics and weather regimes are used to side-adapt the corresponding predictions from up to 5 satellite-derived databases and 2 reanalyses spanning various periods, collectively. It is found that the proposed site-adaptation methods using multiple gridded products are able to attain, on average, a 5 W/m 2 reduction in continuous ranked probability score than that leveraging just a single product. [ABSTRACT FROM AUTHOR]

The interactions between solar forecasting strategies and grid codes have a profound impact on grid integration. In order to develop grid-integration standards, such as the forecast submission requirements or penalty schemes that are in the best interests of both the photovoltaic power plant owners and system operators, various challenges of operational solar forecasting need to be brought forward and addressed adequately. On this point, four very much overlooked technical aspects are identified in this work: (1) gauging the goodness of forecasts, (2) quantifying predictability, (3) forecast downscaling, and (4) hierarchical forecasting. The challenges associated with these aspects are discussed in concert with a case study based on the industry standards issued by regulatory bureaus of the National Energy Administration of the People's Republic of China. [ABSTRACT FROM AUTHOR]

A firm photovoltaic (PV) plant differs from a conventional unconstrained PV plant in terms of its ability to satisfy load demand on a 24/365 basis. Amongst various firm power enablers, overbuilding & proactive curtailment is the most counter-intuitive yet indispensable one. Although the cost-effectiveness of firm PV plants has been studied numerous times, few studies have evaluated the utilization of curtailed energy. To that end, this work advocates using the curtailed energy for hydrogen production, which is not impacted by the intermittency and variability of the curtailed power. A new mathematical optimization model that minimizes the firm kWh premium of the PV–battery–hydrogen hybrid system is put forth. Instead of using just generic modeling for the energy components (i.e., PV, battery, and electrolyzer), refined modeling, which could introduce bilinearity and nonlinearity, is herein considered. To address such optimization difficulty, a new algorithm, which hybridizes the particle swarm optimization and the branch-and-bound method, is proposed. The analysis reveals that the additional inclusion of a hydrogen production system within a firm PV plant is techno-economically attractive, and can lower the curtailment rate by 36%, and the overall firm kWh premium by almost 7%. What this implies is that, under the current market economics, the hydrogen production system becomes entirely free when used with firm PV plants. • A firm photovoltaic–battery–hydrogen hybrid system is proposed. • The hybrid system is able to meet demand 24/365 with 100% certainty. • A hybrid algorithm is proposed for the nonlinear optimization problem. • Power curtailment is necessary to achieve the lowest system cost. [ABSTRACT FROM AUTHOR]

Sensor fault diagnosis is essential to guaranteeing the safety of lithium-ion batteries. To address the general drawbacks of the existing diagnosis methods, including the difficulty in determining the threshold, inability to handle multiple faulty sensors concurrently, and limited capacity in identifying fault modes, a multi-sensor multi-mode fault diagnosis method for lithium-ion battery packs is proposed. The proposed method utilizes time series and discriminative features to accomplish sensor-specific fault detection and fault mode identification. First, a total of 18 general time series features are extracted to characterize the measurements of each sensor during each charge–discharge cycle. Principal component analysis is then used to reduce the high-dimensional feature space to a two-dimensional space, such that fault detection can be carried out with the α -hull algorithm. For the detected faulty samples, a two-layer identification algorithm is designed based on three discriminative features, namely, correlation coefficient, impulse factor, and Hurst coefficient, to identify the specific fault modes. The diagnostics can decouple the information from different types of sensors so that the proposed method can effortlessly isolate current, voltage, and temperature sensors that are concurrently experiencing faults. Ultimately, experimental results from three scenarios, including simultaneous failure of multiple sensors, substantiate the effectiveness and feasibility of the proposed method. [Display omitted] • Sensor faults are detected without establishing models and setting thresholds. • Different types of sensors that malfunction simultaneously are effortlessly isolated. • The fault modes of the faulty samples can be accurately identified. • The proposed method can handle scenarios where multiple sensors fail simultaneously. • The diagnosis results in three scenarios prove the validity of the proposed method. [ABSTRACT FROM AUTHOR]

Orthopedic and dental implants made of β-type Ti alloys have low elastic modulus which can better relieve the stress shielding effects after surgical implantation. Nevertheless, clinical application of β-type Ti alloys is hampered by the insufficient mechanical strength and gradual release of pro-inflammatory metallic ions under physiological conditions. In this study, the β-type Ti-45Nb alloy is subjected to high-pressure torsion (HPT) processing to refine the grain size. After HPT processing, the tensile strength increases from 370 MPa to 658 MPa due to grain boundary strengthening and at the same time, the favorable elastic modulus is maintained at a low level of 61-72 GPa because the single β-phase is preserved during grain refinement. More grain boundaries decrease the work function and facilitate the formation of thicker and less defective passive films leading to better corrosion resistance. In addition, more rapid repair of the passive layer mitigates release of metallic ions from the alloy and consequently, the inflammatory response is suppressed. The results reveal a strategy to simultaneously improve the mechanical and biological properties of metallic implant materials for orthopedics and dentistry. The low modulus Ti-45Nb alloy is promising in addressing the complication of stress shielding induced by biomedical Ti-based materials with too-high elastic modulus. However, its insufficient strength hampers its clinical application, and traditional strengthening via heat treatments will compromise the low elastic modulus. In the current study, we enhanced the ultimate tensile strength of Ti-45Nb from 370 MPa to 658 MPa through grain-refinement strengthening, while the elastic modulus was maintained at a low value (61-72 GPa). Moreover, substrate grain-refinement has been proved to improve the corrosion resistance of Ti-45Nb with reduced inflammatory response both in vitro and in vivo. A relationship between the substrate microstructure and the surface passive layer has been established to explain the beneficial effects of substrate grain-refinement. [Display omitted] [ABSTRACT FROM AUTHOR]

• This work studies the effects of reconciliation on probabilistic forecasts. • A parametric approach is used to generate probabilistic forecasts. • Reconciliation significantly improves the quality of probabilistic forecasts. • Day-ahead and hour-ahead forecasts are reconciled in a geographical hierarchy. Hierarchical forecasting and reconciliation are new to the field of solar engineering. Previous papers in this series, namely, Yang et al. (2017a, 2017b), and Yagli et al. (2019b), discussed various reconciliation techniques for deterministic solar forecasts obtained across spatio-temporal hierarchies. This paper extends the discussion into probability space, and studies how reconciliation can affect the performance of probabilistic forecasting. More specifically, qualities of the parametric predictive distributions before and after reconciliation are compared. Four minimum-trace-based reconciliation techniques are used to reconcile day-ahead and hour-ahead forecasts generated using two datasets: (1) distributed solar power generation for 318 simulated PV systems in California, and (2) satellite-derived irradiance over Arizona. The empirical result shows that reconciliation not only improves the accuracy of point forecasts, but also leads to high-quality predictive distributions in terms of sharpness, calibration, and skill score. Moreover, such improvement is quite general, and does not seem to depend on data, hierarchy structure, nor the underlying forecasting model. [ABSTRACT FROM AUTHOR]

• This review aims at standardizing the forecast verification approaches used in deterministic solar forecasting. • The distribution-oriented forecast verification framework is introduced. • RMSE skill score is recommended to be universally reported in solar forecasting studies. • A series of practical issues during verification are reviewed. The field of energy forecasting has attracted many researchers from different fields (e.g., meteorology, data sciences, mechanical or electrical engineering) over the last decade. Solar forecasting is a fast-growing subdomain of energy forecasting. Despite several previous attempts, the methods and measures used for verification of deterministic (also known as single-valued or point) solar forecasts are still far from being standardized, making forecast analysis and comparison difficult. To analyze and compare solar forecasts, the well-established Murphy–Winkler framework for distribution-oriented forecast verification is recommended as a standard practice. This framework examines aspects of forecast quality, such as reliability, resolution, association, or discrimination, and analyzes the joint distribution of forecasts and observations, which contains all time-independent information relevant to verification. To verify forecasts, one can use any graphical display or mathematical/statistical measure to provide insights and summarize the aspects of forecast quality. The majority of graphical methods and accuracy measures known to solar forecasters are specific methods under this general framework. Additionally, measuring the overall skillfulness of forecasters is also of general interest. The use of the root mean square error (RMSE) skill score based on the optimal convex combination of climatology and persistence methods is highly recommended. By standardizing the accuracy measure and reference forecasting method, the RMSE skill score allows—with appropriate caveats—comparison of forecasts made using different models, across different locations and time periods. [ABSTRACT FROM AUTHOR]

• The most comprehensive validation of satellite-derived and reanalysis irradiance by far. • Hourly gridded irradiance from 8 products are validated at 57 BSRN stations worldwide. • The entire temporal record of BSRN, over 27 years, is used. • Murphy–Winkler distribution-oriented verification framework is emphasized. Gridded solar radiation products, namely satellite-derived irradiance and reanalysis irradiance, are key to the next-generation solar resource assessment and forecasting. Since their accuracies are generally lower than that of the ground-based measurements, providing validation of the gridded solar radiation products is necessary in order to understand their qualities and characteristics. This article delivers a worldwide validation of hourly global horizontal irradiance derived from satellite imagery and reanalysis. The accuracies of 6 latest satellite-derived irradiance products (CAMS-RAD, NSRDB, SARAH-2, SARAH-E, CERES-SYN1deg, and Solcast) and 2 latest global reanalysis irradiance products (ERA5 and MERRA-2) are verified against the complete records from 57 BSRN stations, over 27 years (1992–2018). This scope of validation is unprecedented in the field of solar energy. Moreover, the importance of using distribution-oriented verification approaches is emphasized. Such approaches go beyond the traditional measure-oriented verification approach, and thus can offer additional insights and flexibility to the verification problem. [ABSTRACT FROM AUTHOR]

• Probabilistic solar forecast reconciliation is demonstrated for the first time. • A block bootstrapping technique is used to generate nonparametric predictive distributions. • Data from 318 PV systems in California are used to empirically demonstrate the advantage of this method. • Reconciliation can improve both deterministic and probabilistic forecasts on all levels of the hierarchy. No forecast is complete without understanding it probabilistically. Previously in "Reconciling solar forecasts: Geographical hierarchy" [Sol. Energy 146 (2017) 276–286], four different point forecast reconciliation techniques were demonstrated using simulated data from 318 photovoltaic systems in California. In this paper, I show how to extend those techniques to probabilistic solar forecasting. More specifically, probabilistic forecast reconciliation is performed in a nonparametric framework through block bootstrapping. As compared to the parametric framework, which requires the base forecasts to be characterized by elliptical distributions, the nonparametric framework is not limited by such assumptions. Probabilistic forecast reconciliation not only provides a description of forecast uncertainty, it could also issue optimal point forecasts based on a directive in the form of a statistical functional. In this regard, there is very little reason to favor point forecast reconciliation, or any point forecasting for that matter, in solar energy meteorology. And probabilistic forecast reconciliation, or more generally, probabilistic solar forecasting, should be made default. [ABSTRACT FROM AUTHOR]

Forecast performance of data-driven models depends on the local weather and climate regime, which makes model selection a tedious task for forecast practitioners. Ensemble forecasting, or forecast combination, is beneficial in such cases, in that, forecasts from multiple models are combined to form a final forecast. In ensemble forecasting, additional to the final deterministic-style forecasts, predictive distributions are also available, which can be used by grid operators for better decision-making. Such empirical predictive distributions are useful to represent the uncertainty associated with the forecasts. However, raw ensemble forecasts are often not calibrated, e.g., due to the lack of diversity in the ensemble members. The lack of ensemble spread is known as underdispersion, and it can be ameliorated through post-processing. This study aims to calibrate hourly ensemble clear-sky index forecasts, generated by 20 data-driven models, using both parametric and nonparametric post-processing techniques. Four years of data collected at 7 research-grade sites are used in the empirical part of the paper. Quantitative and qualitative methods are used to evaluate the performance of post-processing techniques in terms of calibration and sharpness. Post-processed ensemble forecasts outperform raw ensemble forecasts under all verification metrics. The proposed parametric post-processing technique, namely, generalized additive models for location, scale and shape, substantially reduces the continuous ranked probability score (CRPS) of the raw ensemble forecasts from 32–59 W/m 2 to 25–45 W/m 2 and quantile score from 16–30 W/m 2 to 13–23 W/m 2. In terms of CRPS skill score, the proposed method achieved 38–58% improvements over a climatology reference. • This work studies the post-processing of ensemble forecasts. • Forecasts from an ensemble of 20 data-driven models are post-processed. • Parametric and nonparametric post-processing techniques are considered. • The performance is verified using quantitative and qualitative measures. • Post-processing with GAMLSS improves the quality of raw ensemble forecasts. [ABSTRACT FROM AUTHOR]

• Ensemble modeling for 1-min diffuse fraction is discussed. • The raw ensemble is post-processed using ensemble model output statistics (EMOS). • EMOS outperforms the best stand-alone models such as Yang2 and Engerer2. • EMOS is superior to simple model blending in terms of CRPS and ignorance score. Separation models split diffuse and direct components of solar radiation from the global horizontal radiation. At the moment, all separation models only issue predictions that are deterministic (as opposed to probabilistic). Since the best prediction is necessarily probabilistic, a parametric post-processing framework called the ensemble model output statistics (EMOS) is introduced in this paper, to make probabilistic predictions. EMOS takes the diffuse fractions predicted by an ensemble of existing 1-min separation models, and outputs a predictive distribution, with parameters optimized by maximum likelihood estimation. Clearly, the EMOS-based separation modeling goes beyond the current literature, in terms of uncertainty quantification. Eight popular separation models from the literature, with different architectures, are used to demonstrate the predictive power of EMOS. Using 1-min high-quality radiometric data from seven stations in the USA and four stations in Europe, it is found that Y ang2 is the best stand-alone model with an average RMSE of 21.8%, in terms of direct normal irradiance prediction, contrasting the 26.3% of the previously reported best model, namely, E ngerer2. On the other hand, the EMOS post-processed predictions have an average RMSE of 20.8%, which is lower than that of the best stand-alone model. Moreover, EMOS is shown superior to simple model averaging, in terms of continuous ranked probability score and ignorance score. [ABSTRACT FROM AUTHOR]

• Popular probabilistic solar forecasting methods are reviewed. • Various methods are compared using a standardized dataset at Folsom, California. • Forecasting methods with camera, satellite and NWP data are superior to the univariate ones. • With the provided data and code, the results shown here can be replicated and compared with ease. The present paper echos a recent data article, "A comprehensive dataset for the accelerated development and benchmarking of solar forecasting methods" [J. Renewable Sustainable Energy 11, 036102 (2019)]. The carefully composed dataset by Pedro, Larson, and Coimbra (PLC) presents a rare opportunity for solar forecasters to develop transparent and reproducible algorithms that can bring incremental contributions to the field. In their original paper, data from four different sources, namely, ground-based measurements, sky-camera images, satellite-imagery features, and numerical weather prediction outputs, were arranged in a machine-learning-ready setup. Subsequently, several benchmarks for deterministic forecasting were set forth, for intra-hour, intra-day, and day-ahead scenarios. Nonetheless, a weather forecast is intrinsically five-dimensional, spanning space, time, and probability. In this regard, five reference methods for probabilistic forecasting: (1) complete-history persistence ensemble, (2) Markov-chain mixture model, (3) ordinary least squares, (4) analog ensemble, and (5) quantile regression, are applied to the PLC dataset. The R code provided in this paper follows the structure of the original Python code precisely, facilitating those solar forecasters who are not familiar with Python but have a statistics background. [ABSTRACT FROM AUTHOR]

As a crucial component of the model chain, which facilitates irradiance-to-power conversion during solar resource assessment and forecasting, separation modeling continues to draw attention in both academia and industry. However, when evaluating even the best separation model today, one can quickly recognize its limited accuracy compared to other energy meteorology models such as transposition models. The task of separating global horizontal irradiance into diffuse and beam components does not seem soluble by any derivative effort aimed at tweaking the existing semi-physical models. As a result, an appealing alternative is to consider end-to-end data-driven models, which have demonstrated predictive capability in scenarios where the volume of data is substantial and the interaction among variables is complex. This work discusses the separation of 1-min irradiance from a data-driven perspective. In this preliminary study, a total of 10 representative data-driven separation models are developed and compared to the state-of-the-art semi-physical models, using a comprehensive 1-min irradiance database that spans five years and covers numerous climate types. The average error of the data-driven models is found to be 15.2% to 22.6% lower than that of the semi-physical models for training locations and 7.9% to 17.6% lower for completely unseen locations. Data-driven models also have significantly lower standard deviations (up to 87.2% even for completely unseen locations), highlighting their robustness. In addition, this work provides a guideline for choosing between data-driven and semi-physical models based on data availability, application needs, computational resources, interpretability, and model adaptability. Furthermore, the study underscores the challenges in accurately predicting the diffuse fraction using available input features and indicates that the incorporation of additional weather-related variables and domain knowledge could enhance the performance of data-driven separation models. • Irradiance separation using end-to-end data-driven techniques. • Thorough comparison of data-driven models with semi-physical models. • Data-driven models demonstrate enhanced overall performance. • In-depth examination of key meteorological features and their relationships. [ABSTRACT FROM AUTHOR]

Since directly measuring beam and diffuse irradiance is not feasible on many occasions, one often has to resort to estimating the beam and diffuse irradiance components from the global irradiance, which is known as separation modeling. Separation modeling is essentially a nonlinear regression problem, with the clearness index being the main input and the diffuse fraction being the output. Hundreds of separation models with various complexities have been proposed, among which the Yang4 model was recently validated using worldwide data as the quasi-universal choice for 1-min data. In this work, Yang4 is further improved by regime-dependent fitting, i.e., fitting a separate set of model coefficients for each climatological regime. Different regimes are determined through clustering of cloud cover frequency, aerosol optical depth, and surface albedo climatology maps. The new Yang5 model is able to outperform its predecessor at the 126 stations tested, covering a wide range of climate types. Overall, the normalized root mean square errors for beam normal irradiance (BNI) and diffuse horizontal irradiance (DHI) of Yang5 are 17.55% and 32.92% on average, as compared to 19.13% and 34.94% for the next best model, namely, Yang4. Furthermore, through conducting pairwise Diebold–Mariano tests, Yang5 is shown superior to Yang4 at 110/126 sites for BNI prediction and 93/126 for DHI. [Display omitted] • A concise summary of recent advances in separation modeling is given. • A regime-dependent version of the Yang4 model is proposed. • Regimes are defined by clustering cloud, aerosol, and albedo climatology. • The new Yang5 model outperforms the quasi-universal Yang4. [ABSTRACT FROM AUTHOR]

Inspecting the literature, much effort has been placed on the verification of irradiance forecasts from numerical weather prediction (NWP) models, as such forecasts are thought to have profound implications on the photovoltaic (PV) power forecasts, which in turn affects grid operators' confidence in integrating such power into the electricity grid. However, perhaps due to the proprietary nature of PV plants and lack of access to state-of-the-art NWP model output, only few have had the chance to conduct head-to-head comparisons of global mesoscale and regional downscaled NWP models, in terms of how their irradiance forecast inaccuracies propagate to PV power forecasts. In this regard, this work presents such a study, in which irradiance and PV power forecasts from the European Centre for Medium-Range Weather Forecasts' High-Resolution (HRES) and Météo-France's Application of Research to Operations at Mesoscale (AROME) models are thoroughly verified against the ground-based measurements from 32 research-grade radiometry stations and 94 actual PV plants in Hungary. A wide range of techniques and case studies concerning verification is herein considered, including variance ratio analysis, Murphy–Winkler decomposition, point-versus-areal verification, and seasonal verification. Despite that the results are too numerous to be summarized in a few sentences, the overarching observation from the verification exercise is that the performance of irradiance forecasts can only be used to infer that of PV power forecasts to a certain extent, which contrasts the conventional wisdom. • Global ECMWF outperforms regional AROME for global horizontal irradiance forecasting. • The advantage of ECMWF is reduced if photovoltaic power forecasting is concerned. • Both numerical weather prediction models have the same accuracy for regional forecasting. • Photovoltaic power forecast errors are the highest in winter. [ABSTRACT FROM AUTHOR]

• Standard of reference in solar forecasting is discussed in a real-time market context. • The choice solely depends on the value of the lag-h autocorrelation. • The optimal convex linear combination of climatology and persistence is recommended. • The statistically derived result is supported through extensive empirical validation. Climatology and persistence are the two most popular naïve reference methods for benchmarking deterministic solar forecasts. Depending on the forecast horizon, the preferred reference methods differ. This brief note derives the general relationship among climatology, persistence, and their optimal convex combination. The value of the lag- h autocorrelation is shown to be the main indicator of choice for the standard of reference. [ABSTRACT FROM AUTHOR]

• Developing an R and Python package to enhance the openness and accessibility of publicly available solar datasets. • Providing access to solar data with multiple types, including sky images, behind-the-meter photovoltaic generation, etc. • Demonstrating the package usage by self-contained scripts. Observational solar data is the foundation of data-driven research in solar power grid integration and power system operations. Compared to other fields in data science, the openness and accessibility of solar data is lacking, which prevents solar data science from catching up with the emerging trends of data science (e.g., deep learning). In this paper, OpenSolar, a package with both R and Python versions, is developed to enhance the openness and accessibility of publicly available solar datasets. The OpenSolar package provides access to multiple types of solar data, primarily from four datasets: (1) the National Renewable Energy Laboratory (NREL) Solar Power Data for Integration Studies dataset, (2) the NREL Solar Radiation Research Laboratory dataset, (3) the Sheffield Solar-Microgen database, and (4) the Dataport database. Unlike other open solar datasets that only contain meteorological data, the four datasets in the OpenSolar package also contain behind-the-meter power data, sky images, and solar power data with satisfactory temporal and spatial resolution and coverage. The overview, quality-control methods, and potential usage of the datasets, in conjunction with sample code implementing the OpenSolar functions, are described in this paper. The package is expected to assist in bridging the gaps between the research fields of solar energy, power systems, and data science. [ABSTRACT FROM AUTHOR]

• An update to the previously published R package "SolarData" is provided. • Functions to read, quality control, and aggregate the BSRN data are included in this update. • As compared to the BSRN-Toolbox, these functions provide additional flexibility in data handling. Previously in "SolarData: An R package for easy access of publicly available solar datasets" [Sol. Energy 171 (2017) A3–A12], the R package SolarData was built for easy access of five publicly available solar datasets. In this update, code for reading data from the Baseline Surface Radiation Network (BSRN), the largest research-grade solar radiation monitoring network, is added to the package. BSRN comprises 66 stations (as of 2019-02) around the globe, which collect 1-min or 3-min radiation data since 1992. These data are stored in the so-called "station-to-archive" files, each containing records from one month and one station, and can be downloaded via ftp. The functions in SolarData v1.1 directly interact with these station-to-archive files, without using the BSRN-Toolbox. It should however be noted that SolarData is not a replacement of BSRN-Toolbox. Instead, it gives R users improved accessibility and freedom in BSRN data manipulation. Like the previous release, all contents herein described are made available on GitHub. [ABSTRACT FROM AUTHOR]