Search

Branko Kerkez, Kris Villez and Eveline I. P. Volcke introduce the Environmental Science: Water Research & Technology themed issue on data-intensive water systems management and operation.

Due to the growing amount of data from in-situ sensors in environmental monitoring, it becomes necessary to automatically detect anomalous data points. Nowadays, this is mainly performed using supervised machine learning models, which need a fully labelled data set for their training process. However, the process of labelling data is typically cumbersome and, as a result, a hindrance to the adoption of machine learning methods for automated anomaly detection. In this work, we propose to address this challenge by means of active learning. This method consists of querying the domain expert for the labels of only a selected subset of the full data set. We show that this reduces the time and costs associated to labelling while delivering the same or similar anomaly detection performances. Finally, we also show that machine learning models providing a nonlinear classification boundary are to be recommended for anomaly detection in complex environmental data sets. © 2020 The Authors, Environmental Modelling & Software, 134, ISSN:1364-8152, ISSN:1873-6726

The aim of this paper is to describe the state-of-the art computer-based techniques for data analysis to improve operation of wastewater treatment plants. A comprehensive review of peer-reviewed papers shows that European researchers have led academic computer-based method development during the last two decades. The most cited techniques are artificial neural networks, principal component analysis, fuzzy logic, clustering, independent component analysis and partial least squares regression. Even though there has been progress on techniques related to the development of environmental decision support systems, knowledge discovery and management, the research sector is still far from delivering systems that smoothly integrate several types of knowledge and different methods of reasoning. Several limitations that currently prevent the application of computer-based techniques in practice are highlighted.

Ammonia control in municipal wastewater treatment plants typically requires maintenance-intensive instrumentation. A low maintenance alternative is sought for small- to medium-scale applications. To this end, a pH-based soft-sensor is proposed to detect ammonia peak load events. This soft-sensor is based on a newly developed technique for qualitative trend analysis and is combined with a rule-based controller. The use of qualitative trend analysis makes this soft-sensor tolerant towards sensor drifts and thereby reduces the maintenance effort. The method allows controlling any process in which relative changes in the measured output are informative about the system output.

Anomaly detection is the process of identifying unexpected data samples in datasets. Automated anomaly detection is either performed using supervised machine learning models, which require a labelled dataset for their calibration, or unsupervised models, which do not require labels. While academic research has produced a vast array of tools and machine learning models for automated anomaly detection, the research community focused on environmental systems still lacks a comparative analysis that is simultaneously comprehensive, objective, and systematic. This knowledge gap is addressed for the first time in this study, where 15 different supervised and unsupervised anomaly detection models are evaluated on 5 different environmental datasets from engineered and natural aquatic systems. To this end, anomaly detection performance, labelling efforts, as well as the impact of model and algorithm tuning are taken into account. As a result, our analysis reveals the relative strengths and weaknesses of the different approaches in an objective manner without bias for any particular paradigm in machine learning. Most importantly, our results show that expert-based data annotation is extremely valuable for anomaly detection based on machine learning.

To this day, obtaining reliable characterization of sludge settling properties remains a challenging and time-consuming task. Without such assessments however, optimal design and operation of secondary settling tanks is challenging and conservative approaches will remain necessary. With this study, we show that automated sludge blanket height registration and zone settling velocity estimation is possible thanks to analysis of images taken during batch settling experiments. The experimental setup is particularly interesting for practical applications as it consists of off-the-shelf components only, no moving parts are required, and the software is released publicly. Furthermore, the proposed multivariate shape constrained spline model for image analysis appears to be a promising method for reliable sludge blanket height profile registration.

The development and validation of methods for fault detection and identification in wastewater treatment research today relies on two important assumptions: {\em (i)} that sensor faults appear at distinct times in different sensors and {\em (ii)} that any given sensor will function near-perfectly for a significant amount of time following installation. In this work, we show that such assumptions are unrealistic, at least for sensors built around an ion-selective measurement principle. Indeed, long-term exposure of sensors to treated wastewater shows that sensors exhibit important fault symptoms that appear simultaneously and with similar intensity. Consequently, our work suggests that focus of research on methods for fault detection and identification should be reoriented towards methods that do not rely on the assumptions mentioned above. This study also provides the very first empirically validated sensor fault model for wastewater treatment simulation and we recommend its use for effective benchmarking of both fault detection and identification methods and advanced control strategies. Finally, we evaluate the value of redundancy for the purpose of remote sensor validation in decentralized wastewater treatment systems.

The control of nitrite-oxidizing bacteria (NOB) challenges the implementation of partial nitritation and anammox (PN/A) processes under mainstream conditions. The aim of the present study was to understand how operating conditions impact microbial competition and the control of NOB in hybrid PN/A systems, where biofilm and flocs coexist. A hybrid PN/A moving-bed biofilm reactor (MBBR; also referred to as integrated fixed film activated sludge or IFAS) was operated at 15 °C on aerobically pre-treated municipal wastewater (23 mgNH4-N·L−1). Ammonium-oxidizing bacteria (AOB) and NOB were enriched primarily in the flocs, and anammox bacteria (AMX) in the biofilm. After decreasing the dissolved oxygen concentration (DO) from 1.2 to 0.17 mgO2·L−1 - with all other operating conditions unchanged - washout of NOB from the flocs was observed. The activity of the minor NOB fraction remaining in the biofilm was suppressed at low DO. As a result, low effluent NO3− concentrations (0.5 mgN·L−1) were consistently achieved at aerobic nitrogen removal rates (80 mgN·L−1·d−1) comparable to those of conventional treatment plants. A simple dynamic mathematical model, assuming perfect biomass segregation with AOB and NOB in the flocs and AMX in the biofilm, was able to qualitatively reproduce the selective washout of NOB from the flocs in response to the decrease in DO-setpoint. Similarly, numerical simulations indicated that flocs removal is an effective operational strategy to achieve the selective washout of NOB. The direct competition for NO2− between NOB and AMX - the latter retained in the biofilm and acting as a “NO2-sink” - was identified by the model as key mechanism leading to a difference in the actual growth rates of AOB and NOB (i.e., μNOB < μAOB in flocs) and allowing for the selective NOB washout. Experimental results and model predictions demonstrate the increased operational flexibility, in terms of variables that can be easily controlled by operators, offered by hybrid systems as compared to solely biofilm systems for the control of NOB in mainstream PN/A applications.HighlightsHybrid PN/A systems provide increased operational flexibility for NOB controlAOB and NOB enrich primarily in the flocs, and AMX in the biofilm (“NO2-sink”)AMX use NO2− allowing to differentiate AOB and NOB growth ratesA decrease in DO or an increase in floc removal leads to selective NOB washout from flocsThe activity of the minor NOB fraction in the biofilm is suppressed at limiting DO

Environmental process modeling is challenged by the lack of high quality data, stochastic variations, and nonlinear behavior. Conventionally, parameter optimization is based on stochastic sampling techniques to deal with the nonlinear behavior of the proposed models. Despite widespread use, such tools cannot guarantee globally optimal parameter estimates. It can be especially difficult in practice to differentiate between lack of algorithm convergence, convergence to a non-global local optimum, and model structure deficits. For this reason, we use a deterministic global optimization algorithm for kinetic model identification and demonstrate it with a model describing a typical batch experiment. A combination of interval arithmetic, reformulations, and relaxations allows globally optimal identification of all (six) model parameters. In addition, the results suggest that further improvements may be obtained by modification of the optimization problem or by proof of the hypothesized pseudo-convex nature of the problem suggested by our results. Display Omitted Objective function bounds for global biokinetic parameter optimization are proven.Global deterministic parameter estimation of a six-parameter model is possible.Globally optimal parameters are obtained to fit the model to experimental data.

Obtaining high quality data collected on wastewater treatment plants is gaining increasing attention in the wastewater engineering literature. Typical studies focus on recognition of faulty data with a given set of installed sensors on a wastewater treatment plant. Little attention is however given to how one can install sensors in such a way that fault detection and identification can be improved. In this work, we develop a method to obtain Pareto optimal sensor layouts in terms of cost, observability, and redundancy. Most importantly, the resulting method allows reducing the large set of possibilities to a minimal set of sensor layouts efficiently for any wastewater treatment plant on the basis of structural criteria only, with limited sensor information, and without prior data collection. In addition, the developed optimization scheme is fast. Practically important is that the number of sensors needed for both observability of all flows and redundancy of all flow sensors is only one more compared to the number of sensors needed for observability of all flows in the studied wastewater treatment plant configurations.

Sensor maintenance is time-consuming and is a bottleneck for monitoring on-site wastewater treatment systems. Hence, we compare maintained and unmaintained sensors to monitor the biological performance of a small-scale sequencing batch reactor (SBR). The sensor types are ion-selective pH, optical dissolved oxygen (DO), and oxidation-reduction potential (ORP) with platinum electrode. We created soft sensors using engineered features: ammonium valley for pH, oxidation ramp for DO, and nitrite ramp for the ORP. Four soft sensors based on unmaintained pH sensors correctly identified the completion of the ammonium oxidation (89-91 out of 107 cycles), about as many times as soft sensors based on a maintained pH sensor (91 out of 107 cycles). In contrast, the DO soft sensor using data from a maintained sensor showed slightly better (89 out of 96 cycles) detection performance than that using data from two unmaintained sensors (77, respectively 82 out of 96 correct). Furthermore, the DO soft sensor using maintained data is much less sensitive to the optimisation of cut-off frequency and slope tolerance than the soft sensor using unmaintained data. The nitrite ramp provided no useful information on the state of nitrite oxidation, so no comparison of maintained and unmaintained ORP sensors was possible in this case. We identified two hurdles when designing soft sensors for unmaintained sensors: i) Sensors' type- and design-specific deterioration affects performance. ii) Feature engineering for soft sensors is sensor type specific, and the outcome is strongly influenced by operational parameters such as the aeration rate. In summary, the results with the provided soft sensors show that frequent sensor maintenance is not necessarily needed to monitor the performance of SBRs. Without sensor maintenance monitoring small-scale SBRs becomes practicable, which could improve the reliability of unstaffed on-site treatment systems substantially.

Monitoring of nitrite is essential for an immediate response and prevention of irreversible failure of decentralized biological urine nitrification reactors. Although a few sensors are available for nitrite measurement, none of them are suitable for applications in which both nitrite and nitrate are present in very high concentrations. Such is the case in collected source-separated urine, stabilized by nitrification for long-term storage. Ultraviolet (UV) spectrophotometry in combination with chemometrics is a promising option for monitoring of nitrite. In this study, an immersible in situ UV sensor is investigated for the first time so to establish a relationship between UV absorbance spectra and nitrite concentrations in nitrified urine. The study focuses on the effects of suspended particles and saturation on the absorbance spectra and the chemometric model performance. Detailed analysis indicates that suspended particles in nitrified urine have a negligible effect on nitrite estimation, concluding that sample filtration is not necessary as pretreatment. In contrast, saturation due to very high concentrations affects the model performance severely, suggesting dilution as an essential sample preparation step. However, this can also be mitigated by simple removal of the saturated, lower end of the UV absorbance spectra, and extraction of information from the secondary, weaker nitrite absorbance peak. This approach allows for estimation of nitrite with a simple chemometric model and without sample dilution. These results are promising for a practical application of the UV sensor as an in situ nitrite measurement in a urine nitrification reactor given the exceptional quality of the nitrite estimates in comparison to previous studies.

Fault detection and identification is challenged by a lack of detailed understanding of process dynamics under anomalous circumstances as well as a lack of historical data concerning rare events in a typical process. Qualitative trend analysis (QTA) techniques provide a way out by focusing on a coarse-grained representation of time series data. Such qualitative representations are valid in a larger set of operating conditions and thus provide a robust way to handle the detection and identification of rare events. Unfortunately, available methods fail when faced with moderate noise levels or result in rather large computational efforts. For this reason, this article provides a novel method for QTA. This leads to dramatic improvements in computational efficiency compared to the previously established shape constrained splines method while the accuracy remains high. © 2015 American Institute of Chemical Engineers AIChE J, 61: 1535–1546, 2015

In nitritation/anammox reactors, several bacterial groups contribute to the overall nitrogen conversion. Knowing the activity of the main bacterial groups, especially of anaerobic ammonium-oxidising bacteria (AMX), is extremely helpful to understand the process and optimise its operation. Mass balances of dissolved compounds such as ammonium, nitrite and nitrate commonly allow the determination of bacterial activities in a nitritation/anammox process, but the activity of heterotrophic bacteria (HET) is usually neglected. However, even in wastewater with a low organic substrate content, heterotrophic denitrification can contribute substantially to nitrogen removal. The goal of this study was to critically evaluate the applicability of mass balances for the determination of the relevant bacterial activities in a nitritation/anammox process with high HET activity. We set up and solved mass balances of different degrees of complexity. Both linear equation systems, with catabolic reactions alone and with balances according to the activated sludge model stoichiometry, do not allow estimation of any of the considered bacterial activities. When kinetic rate expressions are included, it is possible to compute the concentrations of all considered bacterial groups, but the estimation uncertainty is far too high for practical purposes: the relative standard deviation for AMX is 5280%. In a completely autotrophic system, the relative standard deviation for AMX is only 5%, which proves that the high standard deviations are due to the complexity of the nitration–anammox process with HET activity. The high standard deviations of the calculated bacterial concentrations can be significantly reduced by adding an additional mass balance for the total biomass (standard deviation for AMX activity 1210%). The required number of measurements to achieve an acceptable precision, in our example about 600 conversion rate measurements to reach a 50% standard deviation for the AMX concentration, is still far too high though for practical purposes. To conclude, mass balances including kinetics theoretically allow the observation of the bacterial activities in nitritation/anammox reactors with high HET activity. However, the required precision of the calculated conversion rates, the uncertainty of stoichiometric and kinetic parameters and the reactor dynamics (unsteady conditions) make mass balances unsuitable for practical estimation of AMX activity. Due to high frequency and new online instruments, mass balances might become a suitable tool in the future.

Nitrous oxide (N2O) emissions from wastewater treatment contribute significantly to greenhouse gas emissions. They have been shown to exhibit a strong seasonal and daily profile in previously conducted monitoring campaigns. However, only two year-long online monitoring campaigns have been published to date. Based on three monitoring campaigns on three full-scale wastewater treatment plants (WWTPs) with different activated sludge configurations, each of which lasted at least one year, we propose a refined monitoring strategy for long-term emission monitoring with multiple flux chambers on open tanks. Our monitoring campaigns confirm that the N2O emissions exhibited a strong seasonal profile and were substantial on all three plants (1–2.4% of the total nitrogen load). These results confirm that N2O is the most important greenhouse gas emission from wastewater treatment. The temporal variation was more distinct than the spatial variation within aeration tanks. Nevertheless, multiple monitoring spots along a single lane are crucial to assess representative emission factors in flow-through systems. Sequencing batch reactor systems were shown to exhibit comparable emissions within one reactor but significant variation between parallel reactors. The results indicate that considerable emission differences between lanes are to be expected in cases of inhomogeneous loading and discontinuous feeding. For example, N2O emission could be shown to depend on the amount of treated reject water: lanes without emitted

Sensor drift is commonly observed across engineering disciplines, particularly in harsh media such as wastewater. In this study, a novel stabilizing controller for nitrification of high strength ammonia solutions is designed based on online signal derivatives. The controller uses the derivative of a drifting nitrite signal to determine if nitrite-oxidizing bacteria (NOB) are substrate limited or substrate inhibited. To ensure a meaningful interpretation of the derivative signal, the process is excited in a cyclic manner by repeatedly exposing the NOB to substrate-limited and substrate-inhibited conditions. The resulting control system successfully prevented nitrite accumulations for a period of 72 days in a laboratory-scale reactor. Slow disturbances in the form of feed composition changes and temperature changes were successfully handled by the controller while short-term temperature disturbances are shown to pose a challenge to the current version of this controller. Most importantly, we demonstrate that drift-tolerant control for the purpose of process stabilization can be achieved without sensor redundancy by combining deliberate input excitation, qualitative trend analysis, and coarse process knowledge.

Adequate membrane bioreactor operation requires frequent evaluation of the membrane state. A data-driven approach based on principal component analysis (PCA) and fuzzy clustering extracting the necessary monitoring information solely out of transmembrane pressure data was investigated for this purpose. Out of three tested PCA techniques the two functional methods proved useful to cope with noise and outliers as opposed to the common standard PCA, while all of them presented similar capabilities for revealing data trends and patterns. The expert functional PCA approach enabled linking the two major trends in the data to reversible fouling and irreversible fouling. The B-splines approach provided a more objective way for functional representation of the data set but its complexity did not appear justified by better results. The fuzzy clustering algorithm, applied after PCA, was successful in recognizing the data trends and placing the cluster centres in meaningful positions, as such supporting data analysis. However, the algorithm did not allow a correct classification of all data. Factor analysis was used instead, exploiting the linearity of the observed two dimensional trends, to completely split the reversible and irreversible fouling effects and classify the data in a more pragmatic approach. Overall, the tested techniques appeared useful and can serve as the basis for automatic membrane fouling monitoring and control.

Model results are only as good as the data fed as input or used for calibration. Data reconciliation for wastewater treatment modeling is a demanding task, and standardized approaches are lacking. This paper suggests a procedure to obtain high-quality data sets for model-based studies. The proposed approach starts with the collection of existing historical data, followed by the planning of additional measurements for reliability checks, a data reconciliation step, and it ends with an intensive measuring campaign. With the suggested method, it should be possible to detect, isolate, and finally identify systematic measurement errors leading to verified and qualitative data sets. To allow mass balances to be calculated or other reliability checks to be applied, few additional measurements must be introduced in addition to routine measurements. The intensive measurement campaign should be started only after all mass balances applied to the historical data are closed or the faults have been detected, isolated, and identified. In addition to the procedure itself, an overview of typical sources of errors is given.

Wastewater treatment plants (WWTP) are notorious for poor data quality and sensor reliability due to the hostile environment in which the measurement equipment has to function. In this paper, a structured residual approach with maximum sensitivity (SRAMS) based on the redundancy of the measurements is used to detect, identify and reconstruct single and multiple sensor faults in a single reactor for high activity ammonia removal over nitrite (SHARON) process. SRAMS is based on inferences, which are insensitive to the faults in the sensor of interest and sensitive to faults in the other sensors. It is used for four types of sensor failure detection: bias, drift, complete failure and precision degradation. The application of sensor validation shows that single and multiple sensor faults can be detected and that the fault magnitude and fault type can be estimated by the reconstruction scheme. This sensor validation method is not limited by the type or application of the considered sensors. The methodology can thus easily be applied for sensor surveillance of other continuously measuring sensors and analysers.

Recently, a model-based optimisation methodology for SBR operation has been developed and an optimal operation scenario proposed to improve N and P removal in a pilot-scale SBR. In this study, this optimal operation scenario was implemented and evaluated. The results of the implementation showed that the SBR performance was improved by approximately 50 and 40% for total nitrogen and phosphorous removal, respectively, which was better than predicted by the model. However, the long-term SBR performance was found to be unstable, particularly owing to settling problems developed after the implementation. When confronted with reality, the model used for the optimisation of the operation was found to be invalid. The model was unable to predict the nitrite build-up provoked by the optimal operation scenario. These results imply that changing the operation of an SBR system using a model may significantly change the behaviour of the system beyond the (unknown) application domain of the model. This is simply because the mechanistic models currently do not cover all the aspects of activated sludge systems, e.g. settling and adaptation of the microbial community. To further improve model-application practices, expert knowledge (not contained in the models) can be valuable and should be incorporated into model-based process optimisations.

International audience; This paper serves as a problem statement of the issues surrounding uncertainty in wastewater treatment modelling. The paper proposes a structure for identifying the sources of uncertainty introduced during each step of an engineering project concerned with model-based design or optimisation of a wastewater treatment system. It briefly references the methods currently used to evaluate prediction accuracy and uncertainty and discusses the relevance of uncertainty evaluations in model applications. The paper aims to raise awareness and initiate a comprehensive discussion among professionals on model prediction accuracy and uncertainty issues. It also aims to identify future research needs. Ultimately the goal of such a discussion would be to generate transparent and objective methods of explicitly evaluating the reliability of model results, before they are implemented in an engineering decision-making context.

The potential for qualitative representation of trends in the context of process diagnosis and control is evaluated in this paper. The technique for qualitative description of the data series is relatively new to the field of process monitoring and diagnosis and is based on the cubic spline wavelet decomposition of the data. It is shown that the assessed qualitative description of trends can be coupled easily with existing process knowledge and does not demand the user to understand the underlying technique in detail, in contrast to, for instance, multivariate techniques in Statistical Process Control. The assessed links can be integrated straightforwardly into the framework of supervisory control systems by means of look-up tables, expert systems or case-based reasoning frameworks. This in turn allows the design of a supervisory control system leading to fully automated control actions. The technique is illustrated by an application to a pilot-scale SBR.

A methodology based on Principal Component Analysis (PCA) and clustering is evaluated for process monitoring and process analysis of a pilot-scale SBR removing nitrogen and phosphorus. The first step of this method is to build a multi-way PCA (MPCA) model using the historical process data. In the second step, the principal scores and the Q-statistics resulting from the MPCA model are fed to the LAMDA clustering algorithm. This procedure is iterated twice. The first iteration provides an efficient and effective discrimination between normal and abnormal operational conditions. The second iteration of the procedure allowed a clear-cut discrimination of applied operational changes in the SBR history. Important to add is that this procedure helped identifying some changes in the process behaviour, which would not have been possible, had we only relied on visually inspecting this online data set of the SBR (which is traditionally the case in practice). Hence the PCA based clustering methodology is a promising tool to efficiently interpret and analyse the SBR process behaviour using large historical online data sets.

In this paper, two approaches to data mining of time series have been tested and compared. Both methods are based on the wavelet decomposition of data series and allow the localization of important characteristics of a time series in both the time and frequency domain. The first method is a common method based on the analysis of wavelet power spectra. The second approach is new to the applied field of urban water networks and provides a qualitative description of the data series based on the cubic spline wavelet decomposition of the data. It is shown that wavelet power spectra indicate important and basic characteristics of the data but fail to provide detailed information of the underlying phenomena. In contrast, the second method allows the extraction of more and more detailed information that is important in a context of process monitoring and diagnosis

The identification of kinetic models can be simplified via the computation of extents of reaction on the basis of invariants such as stoichiometric balances. With extents, one can identify the structure and the parameters of reaction rates individually, which significantly reduces the number of parameters that need to be estimated simultaneously. So far, extent-based modeling has only been applied to cases where all the extents can be computed from measured concentrations. This generally excludes its application to many biological processes since the number of reactions tends to be larger than the number of measured quantities. This paper shows that, in some cases, such restrictions can be lifted. In addition, in contrast to most extent-based modeling studies that have dealt with simulated data, this study demonstrates the applicability of extent-based model identification using laboratory experimental data.