Search

Understanding the concept of randomness is fundamental for students in introductory statistics courses, but the notion of randomness is deceivingly complex, so it is often emphasized less than the ...

Disinfection is one of the most critical processes for municipal wastewater treatment. However, traditional chemical dosing approaches do not consider how changes in water quality and process opera...

Summary Decentralized waste water treatment facilities monitor many features that are complexly related. The ability to detect the onset of a fault and to identify variables accurately that have shifted because of the fault are vital to maintaining proper system operation and high quality produced water. Various multivariate methods have been proposed to perform fault detection and isolation, but the methods require data to be independent and identically distributed when the process is in control, and most require a distributional assumption. We propose a distribution-free retrospective change-point-detection method for auto-correlated and non-stationary multivariate processes. We detrend the data by using observations from an in-control time period to account for expected changes due to external or user-controlled factors. Next, we perform the fused lasso, which penalizes differences in consecutive observations, to detect faults and to identify shifted variables. To account for auto-correlation, the regularization parameter is chosen by using an estimated effective sample size in the extended Bayesian information criterion. We demonstrate the performance of our method compared with a competitor in simulation. Finally, we apply our method to waste water treatment facility data with a known fault, and the variables identified by our proposed method are consistent with the operators’ diagnosis of the fault's cause.

Polar organic chemical integrative sampler (POCIS) is a passive sampling device that offers many advantages over traditional discrete sampling methods, but quantitative time-weighted average (TWA) concentrations rely heavily on the robustness of sampling rates. The effects of changing chemical concentration exposures on POCIS sampling rates and its ability to operate in an integrative regime were investigated for 12 pesticides across a range of environmentally relevant concentrations. In five independent 21-day experiments, POCIS devices were exposed to these compounds at constant concentrations ranging from 3 to 60 μg/L and multiple pulsed concentrations with maximum peaks ranging from 5 to 150 μg/L (TWA concentrations = 3 to 92 μg/L). For the 21-day exposures to constant and pulsed concentrations, there were no significant differences in the POCIS sampling rates between corresponding TWA concentrations. Similarly, there was no significant effect on POCIS ability to operate in an integrative regime. However, loss of linearity was visible for some replicates when exposed to higher pulsed concentrations over an extended period. Modeling and Freundlich isotherms did not predict sorbent saturation, but the extraction and reconstitution protocol likely contributed to atrazine dissolution and subsequent underestimation of sorbed chemical mass when HLB adsorption exceeded 400 μg.

Objective tools for characterizing materials at the atomic level are often difficult to develop because of the size or structure of the data. Atom probe tomography (APT) is a measurement t...

Counterinsurgencies are a type of violent struggle between state and non-state actors in which one group attempts to gain or maintain influence over a certain portion of the population. When an insurgency (i.e., non-state actor) challenges a host nation (i.e., state actor), often an external counterinsurgent force intervenes. While researchers have categorized insurgencies with social science techniques and United States Army doctrine has established possible counterinsurgency strategies, little research prescribes host nation and counterinsurgent force strength. To this end, we develop a mixed-integer program to provide an estimate of the number of forces required to maximize the probability of a favorable resolution to the counterinsurgent and host nation countries, while minimizing unfavorable resolutions and the number of counterinsurgent deaths. This program integrates: (i) a multivariate piecewise-linear regression model to estimate the number of counterinsurgent deaths each year and (ii) a logistic regression model to estimate the probability of four types of conflict resolution over a 15-year time horizon. Constraints in the model characterize: (i) upper and lower limits on the number of counterinsurgent and host nation forces and their annual rates of increase and decrease, (ii) the characteristics of the type of counterinsurgency, (iii) an estimation of the number of counterinsurgent deaths, and (iv) an estimation of the probability of one of four resolutions. We use Somalia as a case study to estimate how counterinsurgent strategies affect the probability of obtaining each conflict resolution. We conclude that a strategy focusing on building and empowering a stable host nation force provides the highest probability of achieving a positive resolution to the counterinsurgency. Senior leaders can use this information to guide strategic decisions within a counterinsurgency.

As populations and economies expand in regions with changing climates, demand for water can quickly grow beyond what natural supply can sustain. This paper proposes to mitigate shortages, s...

Almost every U.S.-based statistician working on problems motivated by atmospheric science is connected to the statistics program at the National Center for Atmospheric Research (NCAR). Through its ...

Risposta infiammatoria sistemica dopo trattamento chirurgico endoscopico del diverticolo di Zenker.Il diverticolo di Zenker può essere trattato con il laser CO2 o con l’utilizzo di una suturatrice lineare. È stato quindi realizzato questo studio retrospettivo condotto su pazienti affetti da diverticolo di Zenker sottoposti, in elezione, a trattamento chirurgico o con laser CO2 o con suturatrice al fine di analizzare eventuali differenze nella risposta infiammatoria durante il periodo post-operatorio. Sono state misurate la conta dei leucociti e la proteina C-reattiva nel sangue periferico sia il giorno prima dell’operazione, sia in prima, seconda, terza e quinta giornata post-operatoria. È stata quindi eseguita l’analisi statistica utilizzando il Test U di Mann-Whitney. Dei 34 pazienti, 16 sono stati trattati con laser e 18 con suturatrice. Non c’erano differenze di età, sesso e grado ASA tra i due gruppi. La leucocitosi postoperatoria è stata significativamente più modesta nei pazienti trattati con suturatrice rispetto a quelli trattati con laser, e i livelli di proteina C-reattiva in prima, seconda e terza giornata post-operatoria si sono rivelati significativamente più alti nei pazienti trattati con laser. La conta dei leucociti si è normalizzata in terza giornata postoperatoria, invece i livelli di proteina C-reattiva, in entrambi i gruppi, non sono diminuiti, neppure in quinta giornata postoperatoria. Non si sono verificate complicanze infiammatorie, quali polmoniti o mediastiniti. In conclusione, la risposta infiammatoria precoce dopo diverticolomia con laser COZenker’s diverticulum can be treated with a carbon dioxide laser or linear stapling device. A retrospective study on patients undergoing elective surgery for Zenker`s diverticulum with carbon dioxide laser or stapler was performed to analyse possible differences in inflammatory responses during the postoperative period. Leucocyte counts and C-reactive protein levels in peripheral blood were measured before and on days 1, 2, 3 and 5 after the operation. Statistical analysis was performed using the Mann-Whitney U-test. Of 34 patients, 16 were treated by laser and 18 by stapler. Age, sex ratio and ASA grade did not differ between the groups. Postoperative leukocytosis was significantly milder in the stapler group compared with patients who were treated by carbon dioxide laser. The mean C-reactive protein (CRP) level on day 1, 2 and 3 after surgery was significantly higher in the CO

When a spatial regression model that links a response variable to a set of explanatory variables is desired, it is unlikely that the same regression model holds throughout the domain when the spatial domain and dataset are both large and complex. The locations where the trend changes may not be known, and we present here a mixture of regression models approach to identifying the locations wherein the relationship between the predictors and the response is similar; to estimating the model within each group; and to estimating the number of groups. An EM algorithm for estimating this model is presented along with a criterion for choosing the number of groups. Performance of the estimators and model selection are demonstrated through simulation. An example with groundwater depth and associated predictors generated from a large physical model simulation demonstrates the fit and interpretation of the proposed model. R code is provided in the supplementary materials that simulates the scenarios tested herein; implements the method; and reproduces the groundwater depth results. Supplementary materials for this article are available online.

Recent advancements in data-driven process control and performance analysis could provide the wastewater treatment industry with an opportunity to reduce costs and improve operations. However, big data in wastewater treatment plants (WWTP) is widely underutilized, due in part to a workforce that lacks background knowledge of data science required to fully analyze the unique characteristics of WWTP. Wastewater treatment processes exhibit nonlinear, nonstationary, autocorrelated, and co-correlated behavior that (i) is very difficult to model using first principals and (ii) must be considered when implementing data-driven methods. This review provides an overview of data-driven methods of achieving fault detection, variable prediction, and advanced control of WWTP. We present how big data has been used in the context of WWTP, and much of the discussion can also be applied to water treatment. Due to the assumptions inherent in different data-driven modeling approaches (e.g., control charts, statistical process control, model predictive control, neural networks, transfer functions, fuzzy logic), not all methods are appropriate for every goal or every dataset. Practical guidance is given for matching a desired goal with a particular methodology along with considerations regarding the assumed data structure. References for further reading are provided, and an overall analysis framework is presented.

Multivariate statistical monitoring charts are efficient tools for assessing the quality of a process by identifying abnormalities. Most commonly used multivariate monitoring charts, such as the Ho...

Groundwater levels in urban areas are irregularly sampled and not well understood. Using a separable space–time Bayesian Hierarchical Model, we obtain multiple imputations of the missing values to analyze spatial and temporal groundwater level fluctuations in Los Angeles, CA.

Precisely detecting anomalies in process monitoring is beneficial to enhance the operation of the monitored process by avoiding catastrophic failures and reducing maintenance costs. Unsupervised deep learning techniques are increasingly popular because of their capacity to uncover relevant information from large and complex datasets without using labeled data. In this chapter, we review and evaluate the detection performance of recurrent neural networks (RNNs)-based approaches based on a multivariate time series. RNNs are a powerful tool to appropriately model temporal dependencies in multivariate time series data. We first offer a brief overview of RNNs, from the simplest RNNs with no memory states, to sophisticated architectures with several gates and memory components. Particularly, we focus on those that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a gated recurrent unit (GRU). We then present hybrid deep learning models that integrate the desirable features of RNNs and LSTM, which are capable of approximating complex distributions of deep belief networks and restricted Boltzmann machines. We then apply these models with numerous clustering algorithms for uncovering anomalies. We finally demonstrate these methods on real measurements of effluents from a coastal municipal wastewater treatment plant in Saudi Arabia.

Developing efficient anomaly detection and isolation schemes that offer early detection of potential anomalies in the monitored process and identify and isolate the source of the detected anomalies is indispensable to monitor process operations in an efficient manner. This will further enhance availability, operation reliability, and profitability of monitored processes and reduce manpower costs. This book is mainly devoted to data-driven fault detection and isolation methods based on multivariate statistical monitoring techniques and deep learning methods. In this chapter, conclusions and further research directions are drawn.

Fast-paced developments in data acquisition, instrumentation technology and the era of the Internet-of-Things have resulted in large amounts of data produced by modern industrial processes. The ability to extract useful information from these multivariate datasets has vital benefits that could be utilized in process monitoring. In the absence of a physics-based process model, data-driven approaches such as latent variable modeling have proved to be practical for process monitoring over the past four decades. The aim of this chapter is to review and show the challenges in multivariate process monitoring based on linear models. Specifically, after presenting the limitations of the full-rank regression model, we provide a brief overview of linear latent variable models such as principal component analysis, principal component regression, and partial least squares regression. To deal with dynamic systems, we present dynamic extensions of these methods that capture both static and dynamic features in multivariate processes. We then provide a brief overview of univariate monitoring schemes, such as exponentially-weighted moving average and cumulative sum and generalized likelihood ratio monitoring schemes and their multivariate counterparts. To apply such tools to multivariate data, we employ appropriate multivariate dimension-reduction techniques according to the features of a process, and we use monitoring schemes to monitor more informative variables in a lower dimension. Next, we aim to identify which process variables contribute to abnormal change; conventional contribution plots and radial visualization tool are briefed. Lastly, the effectiveness of the presented inferential modeling techniques is assessed using simulated data. We also present a study on monitoring influent measurements at a water resource recovery facility. Finally, we discuss limitations of the presented monitoring approaches and give some possible directions to rectify these limitations.

In this chapter, first we provide an overview of some of the shallow-machine learning approaches used in anomaly detection and outlier detection in data mining, namely data clustering techniques. Then, we give a brief description of two frequently used unsupervised machine learning algorithms for one-class classification or detection, namely one-class SVM and support vector data description (SVDD). Particular attention is paid to deep learning models. We present the commonly used deep learning models based on autoencoders (Variational Autoencoder, Denoising Autoencoder, and Contrastive Autoencoder), probabilistic models (Boltzmann Machine and Restricted Boltzmann Machine) and deep neural models (Deep Belief Network and Deep Boltzmann Machine), and we show their capacity and limitations. Finally, we merge the desirable properties of shallow learning approaches, such as one-class support vector machine and k-nearest neighbors and unsupervised deep-learning approaches to develop more sophisticated and efficient monitoring techniques.

Detecting anomalies is crucially important for improving the operation, reliability, and profitability of complex industrial processes. Traditional linear data-driven methods, such as the principal component analysis (PCA) and partial least squares (PLS) method, are extensively exploited for detecting anomalies in multivariate correlated processes. Since most of the data observed in practical applications are innately nonlinear, the development of models able to learn such nonlinearity are indispensable. In this chapter, in order to handle nonlinearity, we use nonlinear latent variable regression (LVR) modeling methods, which are powerful tools for processing nonlinearities. First, we use nonlinear functions using polynomials an adaptive network-based fuzzy-inference system as an inner model of the LVR model (i.e., nonlinear relation between latent variables and output). We then offer a brief overview of nonlinear LVR-based monitoring approaches and how they can be used for anomaly detection. We also present an alternative for dealing with nonlinearities in-process data by using kernel PCA, which captures the nonlinear features in high-dimensional feature spaces via nonlinear kernel functions. Lastly, the methods presented are applied to simulated synthetic data, plug flow reactor data, and real data from a wastewater treatment plant located in Saudi Arabia.

Data acquired from industrial processes, usually via sensors, are generally noisy, correlated in time and nonstationary; this makes the implementation of the monitoring process difficult, as most techniques are designed for Gaussian and uncorrelated observations. As conventional monitoring methods, their efficiency may be significantly affected by typical uncertainties in industrial processes. Assumptions of Gaussianity, dependence in time, and stationarity are typically not verified in industrial processes. These properties make wavelet-based fault detection approaches especially appropriate. Wavelet methods are also helpful when the characteristics of the fault are unknown. This chapter discusses wavelet-based monitoring approaches that are flexible and designed with fewer structural assumptions. In this chapter, we present a brief overview of wavelets and their desirable characteristics, as well as the discrete wavelet transform. We then assess the effect of violating these assumptions (in addition to the effect of noise levels), based on the performances of the univariate monitoring methods, provide an overview of the univariate wavelet-based technique. And then discuss and illustrate the wavelet-based multivariate extension of LVR methods. At the end of the chapter, the methods are demonstrated on distillation column data.

Luminal succinate (Suc) has been reported to activate its receptor Sucnr1, stimulate renin release and increase BP in certain contexts. NaDC1 (Na + dicarboxylate cotransporter), located only on apical membrane in the proximal tubule of the kidney, reabsorbs filtered citrate and Suc, and is upregulated in acidosis. NaDC1 regulation is key in preventing stones and in maintaining acid-base homeostasis; but its role in BP regulation is not known. We postulate that luminal Suc alters BP. Our purpose was to examine the role of NaDC1, luminal Suc, and acidosis in BP regulation. To address these issues, we used NaDC1 KO (knock out) and WT (wild type) mice on normal diet or 72 hr acid diet (AD). Acidosis should lower luminal Suc due to upregulation of NaDC1 in WT mice. AD was associated with statistically significant BP decreases in both male and female WT but not in NaDC1 KO. Clearance studies compared Suc infused (SI) with non-infused NaDC1 KO and WT. ANOVA showed borderline increases between some groups: WT males on AD, BP increased from 71.48 ± 2.30 to 81.63 ± 2.16 with SI; and NaDC1 KO females on normal diet BP increased from 86.20 ± 3.98 to 96.48 ± 3.39 with SI. KO was only associated with increased BP in males on AD (p=0.05). Recently Khamaysi et al (JASN 2019) found only activity-dependent BP was altered by Suc. So, in additional studies we used telemetry (BP/T) for 24/7 monitoring of NaDC1 KO vs WT. On normal diets, mean arterial pressure (MAP) was significantly higher in KO than in WT. MAP at 9 pm: KO 122.44 ± 1.85 vs WT 110.40 ± 2.59, p

A novel application for outlier detection is in ground deformation monitoring. During any type of underground construction in urban settings, sensors are placed on the ground surface to monitor the vertical displacement with the goal of ensuring that there is no substantial heaving or settling of the ground. As a result, a large spatial-temporal data set is produced, but the sensors are often very sensitive, and spurious readings are commonly observed, resulting in both random and systematic outliers. In this work, we present a novel, fast spatial-temporal quality control procedure that is designed to remove these spurious readings prior to subsequent ground deformation monitoring. First, a robust kriging model is applied to the spatial ground deformations at each time point to remove systematic errors; next, an exponential moving average model is applied to the time series of ground deformations at each station to remove random outliers. A case study using ground deformation data when four subway tunnels are bored under a railyard in Queens, New York is used to illustrate the methodology. Methods used to construct outlier bounds are described, and the accuracy of our outlier detection approach is evaluated by calculating the percentages of outliers detected in an introduced artificial outlier set.

Mainstream anaerobic treatment of domestic wastewater is a promising energy-generating treatment strategy; however, such reactors operated in colder regions are not well characterized. Performance data from a pilot-scale, multiple-compartment anaerobic reactor taken over 786 days were subjected to comprehensive statistical analyses. Results suggest that chemical oxygen demand (COD) was a poor proxy for organics in anaerobic systems as oxygen demand from dissolved inorganic material, dissolved methane, and colloidal material influence dissolved and particulate COD measurements. Additionally, univariate and functional boxplots were useful in visualizing variability in contaminant concentrations and identifying statistical outliers. Further, significantly different dissolved organic removal and methane production was observed between operational years, suggesting that anaerobic reactor systems may not achieve steady-state performance within one year. Last, modeling multiple-compartment reactor systems will require data collected over at least two years to capture seasonal variations of the major anaerobic microbial functions occurring within each reactor compartment.

Which of two competing continuous forecasts is better? This question is often asked in forecast verification, as well as climate model evaluation. Traditional statistical tests seem to be well suited to the task of providing an answer. However, most such tests do not account for some of the special underlying circumstances that are prevalent in this domain. For example, model output is seldom independent in time, and the models being compared are geared to predicting the same state of the atmosphere, and thus they could be contemporaneously correlated with each other. These types of violations of the assumptions of independence required for most statistical tests can greatly impact the accuracy and power of these tests. Here, this effect is examined on simulated series for many common testing procedures, including two-sample and paired t and normal approximation z tests, the z test with a first-order variance inflation factor applied, and the newer Hering–Genton (HG) test, as well as several bootstrap methods. While it is known how most of these tests will behave in the face of temporal dependence, it is less clear how contemporaneous correlation will affect them. Moreover, it is worthwhile knowing just how badly the tests can fail so that if they are applied, reasonable conclusions can be drawn. It is found that the HG test is the most robust to both temporal dependence and contemporaneous correlation, as well as the specific type and strength of temporal dependence. Bootstrap procedures that account for temporal dependence stand up well to contemporaneous correlation and temporal dependence, but require large sample sizes to be accurate.

The concept of big data has caught the attention of business leaders. However, there is still widespread confusion in industry as to how to treat such data. We describe one such encounter with big data from an industrial parts supplier who was concerned with unexpected variability in its prices. Unable to discern trends in the data, a point of contact for the supplier worked with us to explore this concern. Analysis showed that customers at different branches of the company were experiencing significantly different levels of price variation, and that some customers within a specific branch were being offered products at widely varying prices, which were apparently uncorrelated with the quantity of products purchased. Such behaviors are unacceptable to end customers, and rectification of these behaviors has led to increased customer satisfaction for this company. Furthermore, we were able to demonstrate general methodologies to help the company with future analyses.

As part of an energy-reduction study at remote sites, we explore a power system comprised of hybrid renewable energy technologies, specifically, photovoltaic cells, battery storage, and diesel generators. An optimization model determines the design and dispatch strategy of the power system to meet load off grid, such as at a military forward operating base. The model alternately uses two types of load data from government agencies, simulated and observed, to assess the effects of these inputs. Because the latter data set contains errors and is incomplete, we detail the process of cleaning and imputing it to provide a year’s worth in hourly increments for two forward operating bases in Afghanistan. We then construct an approximation of a realistic 600-soldier camp load from the full year of observed data. We compare the design and dispatch output from the optimization model using the simulated and constructed (observed) data sets and demonstrate that the results can differ. We investigate the characteristics of load that influence the optimization model’s behavior regarding the design and dispatch strategy and show that mean load has a more pronounced effect than its shape. In addition, the photovoltaic cells are often used to help the generators run more efficiently, especially under load variability.

The transport properties of a disordered two-dimensional (2D) honeycomb lattice are examined numerically using the spectral approach to the 2D percolation problem, characterized by an Anderson-type Hamiltonian. In our model, disorder is represented by two parameters: a distribution of random on-site energies ${\ensuremath{\epsilon}}_{i}$ (positional disorder) and a concentration of doping energies $p$ (substitutional disorder). The results indicate the existence of extended energy states for nonzero disorder and the emergence of a transition towards localized behavior for critical doping concentration ${n}_{D}g0.3%$, in agreement with the experimentally observed metal-to-insulator transition in a graphene sheet doped with hydrogen.

There is an ever-evolving need in the field of in vitro toxicology to improve the quality of experimental design, ie, from ill-defined cell cultures to well-characterized cytotoxicological models. This evolution is especially important as environmental health scientists begin to rely more heavily on cell culture models in pulmonary toxicology studies. The research presented in this study analyzes the differences and similarities of cells derived from two different depths of the human lung with varying phenotypes. We compared cell cycle and antioxidant-related mRNA and protein concentrations of primary, transformed, and cancer-derived cell lines from the upper and lower airways. In all, six of the most commonly used cell lines reported in in vitro toxicology research papers were included in this study (ie, PTBE, BEAS-2B, A549, PSAE, Met-5A, and Calu-3). Comparison of cell characteristics was accomplished through molecular biology (q-PCR, ELISA, and flow cytometry) and microscopy (phase and fluorescence) techniques as well as cellular oxidative stress endpoint analyses. After comparing the responses of each cell type using statistical analyses, results confirmed significant differences in background levels of cell cycle regulators, inherent antioxidant capacity, pro-inflammatory status, and differential toxicological responses. The analyzed data improve our understanding of the cell characteristics, and in turn, aids in more accurate interpretation of toxicological results. Our conclusions suggest that in vitro toxicology studies should include a detailed cell characterization component in published papers.

Biological time series datasets provide an unparalleled opportunity to investigate regional and global changes in the marine environment. Baleen whales are long-lived sentinel species and an integral part of the marine ecosystem. Increasing anthropogenic terrestrial and marine activities alter ocean systems, and such alterations could change foraging and feeding behavior of baleen whales. In this study, we analyzed δ13C and δ15N of baleen whale earplugs from three different species (N = 6 earplugs, n = 337 laminae) to reconstruct the first continuous stable isotope profiles with a six-month resolution. Results of our study provide an unprecedented opportunity to assess behavioral as well as ecological changes. Abrupt shifts and temporal variability observed in δ13C and δ15N profiles could be indicative of behavior change such as shift in foraging location and/or trophic level in response to natural or anthropogenic disturbances. Additionally, five out of six individuals demonstrated long-term declining trends in δ13C profiles, which could suggest influence of emission of depleted 13CO2 from fossil fuel combustion referred to as the Suess effect. After adjusting the δ13C values of earplugs for the estimated Suess effect and re-evaluating δ13C profiles, significant decline in δ13C values as well as different rate of depletion suggest contribution of other sources that could impact δ13C values at the base of the food web.

Decentralized wastewater treatment (WWT) can be an energy and resource efficient alternative to the traditional, centralized WWT paradigm for water-stressed communities. However, to operate economically, decentralized facilities do not typically have a WWT operator on-site full-time, so a real-time monitoring scheme is needed to quickly detect system faults and isolate the features associated with or affected by faults to ensure adequate treated water quality. Data collected from WWT facilities exhibit temporal dependence and experience natural fluctuations in the mean due to environmental and operator-controlled factors, violating the assumptions of many existing fault detection and isolation (FD&I) methods. To address this, we develop a complete data-driven FD&I method tuned to handle the unique features of WWT data that can be run in real-time and illustrate how it performs with data from a decentralized WWT facility in Golden, Colorado, USA. Enhanced visualization techniques are designed to assist operators in identifying features associated with the fault. We present three case studies with known faults and demonstrate how this method can aid operators in detecting and diagnosing the cause of a fault more quickly.

Hydro-thermal scheduling is the problem of finding an optimal dispatch of power plants in a system containing both hydro and thermal plants. Since hydro plants are able to store water over long time periods, and since future inflows are uncertain due to precipitation, the resulting multi-stage stochastic optimization problem becomes challenging to solve. Several solution methods have been developed over the past few decades to compute practically useful operation policies. One of these methods is stochastic dual dynamic programming (SDDP). SDDP poses strong restrictions on the forecasting method generating the necessary inflow scenarios. In this context, the current state-of-the-art in forecasting are periodic autoregressive (PAR) models. We present a new forecasting model for hydro inflows that incorporates spatial information, i.e. , inflow information from neighboring reservoirs of the system, and that also satisfies the restrictions posed by SDDP. We benchmark our model against a PAR model that is similar to the one currently used in Brazil. Three multi-reservoir basins in Brazil serve as a case study for the comparison. We show that our approach outperforms the benchmark PAR model and present the root mean squared error (RMSE) as well as the seasonally-adjusted coefficient of efficiency (SACE) for each reservoir modeled. The overall decrease in RMSE is 8.29 percent using our approach for one month-ahead forecasts. The decrease in RMSE is achieved without additional data collection while only adding 11.8 percent more state variables for the SDDP algorithm.

Counterinsurgencies are conflicts where an insurgent organization conducts violence to replace or influence a recognized government. Furthering our understanding of the conditions that influence violence in different types of counterinsurgencies is important to government leaders who must deploy scarce resources efficiently. Subject matter experts (SMEs) have developed classification schemes that divide counterinsurgencies into similar groups, but no data-driven methods have ever been developed. Using the robust partitioning around medoids (PAM) algorithm, we cluster counterinsurgencies based on distances among independent variables measured on each counterinsurgency. We apply several criteria for choosing the optimal number of clusters, and then we take these groups of counterinsurgencies and build regression models for counterinsurgent deaths, an annual measure of conflict status. We evaluate these schemes using cross-validation to select the grouping whose regression models best predict counterinsurgent deaths. This approach produces a set of data-driven clusters whose predictive ability is similar to the best existing SME classification scheme, but reduces error in the assignment of a new counterinsurgency to a cluster.

Multivariate statistical methods for online process monitoring have been widely applied to chemical, biological, and engineered systems. While methods based on principal component analysis (PCA) are popular, more recently kernel PCA (KPCA) and locally linear embedding (LLE) have been utilized to better model nonlinear process data. Additionally, various forms of dynamic and adaptive monitoring schemes have been proposed to address time-varying features in these processes. In this analysis, we extend a common simulation study in order to account for autocorrelation and nonstationarity in process data and comprehensively compare the monitoring performances of static, dynamic, adaptive, and adaptive–dynamic versions of PCA, KPCA, and LLE. Furthermore, we evaluate a nonparametric method to set thresholds for monitoring statistics and compare results with the standard parametric approaches. We then apply these methods to real-world data collected from a decentralized wastewater treatment system during normal and abnormal operations. From the simulation study, adaptive–dynamic versions of all three methods generally improve results when the process is autocorrelated and nonstationary. In the case study, adaptive–dynamic versions of PCA, KPCA, and LLE all flag a strong system fault, but nonparametric thresholds considerably reduce the number of false alarms for all three methods under normal operating conditions.

In this work, a statistical stability metric and novel hybrid statistical-machine learning ammonia forecasting model are developed to improve the accuracy and precision of municipal wastewater treatment. Aeration for biological nutrient removal is typically the largest energy expense for municipal wastewater treatment plants (WWTP). Ammonia-based aeration control (ABAC) is one approach designed to minimize excessive aeration by adjusting air blower output from online ammonia measurements rather than from a dissolved oxygen (DO) sensor, which is the conventional aeration control approach. We propose a quantitative stability metric, Total Sample Variance, to compare system-wide variability of competing aeration control strategies. Using this metric, the performance of traditional DO and ABAC control strategies with varying setpoints and control parameters were compared in a medium-sized WWTP, and the most stable strategy was identified and implemented at the facility. To further improve ABAC performance, ammonia forecasting models were constructed using both statistical and machine learning to improve the accuracy of the aeration control system. Diurnal, diurnal-linear, artificial neural network (ANN), and hybrid diurnal-linear-ANN forecasting models were trained on real-time plant-wide process data. The diurnal-linear and diurnal-linear-ANN forecasts were found to most accurately forecast ammonia; improving upon the existing ammonia measurement by up to 32% and 46%, respectively, whereas the ANN model forecast was only able to improve by up to 8%. This work demonstrates the ease and flexibility of integrating statistics and machine learning methods for developing new treatment models in conventional WWTP for features in full-scale conventional activated sludge systems.