Search

The performance of conceptual catchment runoff models may highly depend on the specific choice of calibration methods made by the user. Particle Swarm Optimization (PSO) and Differential Evolution (DE) are two well-known families of Evolutionary Algorithms that are widely used for calibration of hydrological and environmental models. In the present paper, five DE and five PSO optimization algorithms are compared regarding calibration of two conceptual models, namely the Swedish HBV model (Hydrologiska Byrans Vattenavdelning model) and the French GR4J model (modèle du Génie Rural à 4 paramètres Journalier) of the Kamienna catchment runoff. This catchment is located in the middle part of Poland. The main goal of the study was to find out whether DE or PSO algorithms would be better suited for calibration of conceptual rainfall-runoff models. In general, four out of five DE algorithms perform better than four out of five PSO methods, at least for the calibration data. However, one DE algorithm constantly performs very poorly, while one PSO algorithm is among the best optimizers. Large differences are observed between results obtained for calibration and validation data sets. Differences between optimization algorithms are lower for the GR4J than for the HBV model, probably because GR4J has fewer parameters to optimize than HBV. [ABSTRACT FROM AUTHOR]

Water temperature is one of the most important indicators of aquatic system, and accurate forecasting of water temperature is crucial for rivers. It is a complex process to accurately predict stream water temperature as it is impacted by a lot of factors (e.g., meteorological, hydrological, and morphological parameters). In recent years, with the development of computational capacity and artificial intelligence (AI), AI models have been gradually applied for river water temperature (RWT) forecasting. The current survey aims to provide a systematic review of the AI applications for modeling RWT. The review is to show the progression of advances in AI models. The pros and cons of the established AI models are discussed in detail. Overall, this research will provide references for hydrologists and water resources engineers and planners to better forecast RWT, which will benefit river ecosystem management. [ABSTRACT FROM AUTHOR]

• New logistic regression model is proposed for stream temperature simulations. • Air temperature, flow and declination of the Sun are used as input variables. • Proposed model is simple and may easily be applied in practice. • Proposed model is tested on six streams located in various morphological conditions. • Proposed model outperforms other logistic regression models and semi-physical model. Among various stream temperature models those based on nonlinear regression frequently attract attention due to their simplicity and small number of required variables. Among such approaches the logistic regression model developed twenty years ago for weekly data is still widely used in various scientific studies that require quick and simple calculation of stream water temperature. The model has been modified a number of times in recent years to capture the relationship between daily stream water temperatures, air temperatures and discharge. In this study, we propose further modifications of the logistic regression model that do not require any additional variables that may be hard to measure. The proposed models capture the relationship between the stream temperature and the declination of the Sun, the air temperature and the discharge from a number of recent observations. The proposed approaches are tested on six rivers located in diverse orographic conditions of temperate climate zones of Europe and USA. Although the proposed models remain very simple, their performances are competitive against the performances of more advanced semi-physical or data-driven models. [ABSTRACT FROM AUTHOR]

A number of physical or data-driven models have been proposed to evaluate stream water temperatures based on hydrological and meteorological observations. However, physical models require a large amount of information that is frequently unavailable, while data-based models ignore the physical processes. Recently the air2stream model has been proposed as an intermediate alternative that is based on physical heat budget processes, but it is so simplified that the model may be applied like data-driven ones. However, the price for simplicity is the need to calibrate eight parameters that, although have some physical meaning, cannot be measured or evaluated a priori . As a result, applicability and performance of the air2stream model for a particular stream relies on the efficiency of the calibration method. The original air2stream model uses an inefficient 20-year old approach called Particle Swarm Optimization with inertia weight. This study aims at finding an effective and robust calibration method for the air2stream model. Twelve different optimization algorithms are examined on six different streams from northern USA (states of Washington, Oregon and New York), Poland and Switzerland, located in both high mountains, hilly and lowland areas. It is found that the performance of the air2stream model depends significantly on the calibration method. Two algorithms lead to the best results for each considered stream. The air2stream model, calibrated with the chosen optimization methods, performs favorably against classical streamwater temperature models. The MATLAB code of the air2stream model and the chosen calibration procedure (CoBiDE) are available as Supplementary Material on the Journal of Hydrology web page. [ABSTRACT FROM AUTHOR]

In recent years sampling approaches have been used more widely than optimization algorithms to find parameters of conceptual rainfall–runoff models, but the difficulty of calibration of such models remains in dispute. The problem of finding a set of optimal parameters for conceptual rainfall–runoff models is interpreted differently in various studies, ranging from simple to relatively complex and difficult. In many papers, it is claimed that novel calibration approaches, so-called metaheuristics, outperform the older ones when applied to this task, but contradictory opinions are also plentiful. The present study aims at calibration of two simple lumped conceptual hydrological models, HBV and GR4J, by means of a large number of metaheuristic algorithms. The tests are performed on four catchments located in regions with relatively similar climatic conditions, but on different continents. The comparison shows that, although parameters found may somehow differ, the performance criteria achieved with simple lumped models calibrated by various metaheuristics are very similar and differences are insignificant from the hydrological point of view. However, occasionally some algorithms find slightly better solutions than those found by the vast majority of methods. This means that the problem of calibration of simple lumped HBV or GR4J models may be deceptive from the optimization perspective, as the vast majority of algorithms that follow a common evolutionary principle of survival of the fittest lead to sub-optimal solutions. [ABSTRACT FROM PUBLISHER]

In the mid 1990s two landmark metaheuristics have been proposed: Particle Swarm Optimization and Differential Evolution. Their initial versions were very simple, but rapidly attracted wide attention. During the last quarter century hundreds of variants of both optimization algorithms have been proposed and applied in almost any field of science or engineering. However, no broader comparison of performance between both families of methods has been presented so far. In the present paper ten Particle Swarm Optimization and ten Differential Evolution variants, from historical ones from the 1990s up to the most recent ones from 2022, are compared on numerous single-objective numerical benchmarks and 22 real-world problems. On average Differential Evolution algorithms clearly outperform Particle Swarm Optimization ones. Such advantage of Differential Evolution over Particle Swarm Optimization is in contradiction with popularity: In the literature Particle Swarm Optimization algorithms are two–three times more frequently used than Differential Evolution ones. Problems for which Particle Swarm Optimization performs better than Differential Evolution do exist but are relatively few. Although this result may be an effect of the choice of specific variants, experimental settings or problems used for comparison, some re-consideration of algorithmic philosophy may be needed for Particle Swarm Optimization variants to make them more competitive. [ABSTRACT FROM AUTHOR]

Artificial neural networks (ANNs) become widely used for runoff forecasting in numerous studies. Usually classical gradient-based methods are applied in ANN training and a single ANN model is used. To improve the modelling performance, in some papers ensemble aggregation approaches are used whilst in others, novel training methods are proposed. In this study, the usefulness of both concepts is analysed. First, the applicability of a large number of population-based metaheuristics to ANN training for runoff forecasting is tested on data collected from four catchments, namely upper Annapolis (Nova Scotia, Canada), Biala Tarnowska (Poland), upper Allier (France) and Axe Creek (Victoria, Australia). Then, the importance of the search for novel training methods is compared with the importance of the use of a very simple ANN ensemble aggregation approach. It is shown that although some metaheuristics may slightly outperform the classical gradient-based Levenberg-Marquardt algorithm for a specific catchment, none performs better for the majority of the tested ones. One may also point out a few metaheuristics that do not suit ANN training at all. On the other hand, application of even the simplest ensemble aggregation approach clearly improves the results when the ensemble members are trained by any suitable algorithms.EDITOR D. Koutsoyiannis; ASSOCIATE EDITOR E. Toth [ABSTRACT FROM AUTHOR]

Summary A number of methods have been proposed for the prediction of streamwater temperature based on various meteorological and hydrological variables. The present study shows a comparison of few types of data-driven neural networks (multi-layer perceptron, product-units, adaptive-network-based fuzzy inference systems and wavelet neural networks) and nearest neighbour approach for short time streamwater temperature predictions in two natural catchments (mountainous and lowland) located in temperate climate zone, with snowy winters and hot summers. To allow wide applicability of such models, autoregressive inputs are not used and only easily available measurements are considered. Each neural network type is calibrated independently 100 times and the mean, median and standard deviation of the results are used for the comparison. Finally, the ensemble aggregation approach is tested. The results show that simple and popular multi-layer perceptron neural networks are in most cases not outperformed by more complex and advanced models. The choice of neural network is dependent on the way the models are compared. This may be a warning for anyone who wish to promote own models, that their superiority should be verified in different ways. The best results are obtained when mean, maximum and minimum daily air temperatures from the previous days are used as inputs, together with the current runoff and declination of the Sun from two recent days. The ensemble aggregation approach allows reducing the mean square error up to several percent, depending on the case, and noticeably diminishes differences in modelling performance obtained by various neural network types. [ABSTRACT FROM AUTHOR]

Abstract: Nature-inspired metaheuristics found various applications in different fields of science, including the problem of artificial neural networks (ANN) training. However, very versatile opinions regarding the performance of metaheuristics applied to ANN training may be found in the literature. Both nature-inspired metaheuristics and ANNs are widely applied to various geophysical and environmental problems. Among them the water temperature forecasting in a natural river, especially in colder climate zones where the seasonality plays important role, is of great importance, as water temperature has strong impact on aquatic life and chemistry. As the impact of possible future climate change on water temperature is not trivial, models are needed to allow projection of streamwater temperature based on simple hydro-meteorological variables. In this paper the detailed comparison of the performance of nature-inspired optimization methods and Levenberg–Marquardt (LM) algorithm in ANNs training is performed, based on the case study of water temperature forecasting in a natural stream, namely Biala Tarnowska river in southern Poland. Over 50 variants of 22 various metaheuristics, including a large number of Differential Evolution, as well as some Particle Swarm Optimization, Evolution Strategies, multialgorithms and Direct Search methods are compared with LM algorithm on ANN training for the described case study. The impact of population size and some control parameters of particular metaheuristics on the ANN training performance are verified. It is found that despite widely claimed large improvement in nature-inspired methods during last years, the vast majority of them are still outperformed by LM algorithm on the selected problem. The only methods that, based on this case study, seem competitive to LM algorithm in terms of the final performance (but not speed) are Differential Evolution algorithms that benefit from the concept of Global and Local neighborhood-based mutation operators. The streamwater forecasting performance of the neural networks is adequate, the major prediction errors are related to the river freezing and melting processes that occur during winter in the mountainous catchment under study. [Copyright &y& Elsevier]

Summary: Artificial neural networks (ANNs) becomes very popular tool in hydrology, especially in rainfall–runoff modelling. However, a number of issues should be addressed to apply this technique to a particular problem in an efficient way, including selection of network type, its architecture, proper optimization algorithm and a method to deal with overfitting of the data. The present paper addresses the last, rarely considered issue, namely comparison of methods to prevent multi-layer perceptron neural networks from overfitting of the training data in the case of daily catchment runoff modelling. Among a number of methods to avoid overfitting the early stopping, the noise injection and the weight decay have been known for about two decades, however only the first one is frequently applied in practice. Recently a new methodology called optimized approximation algorithm has been proposed in the literature. Overfitting of the training data leads to deterioration of generalization properties of the model and results in its untrustworthy performance when applied to novel measurements. Hence the purpose of the methods to avoid overfitting is somehow contradictory to the goal of optimization algorithms, which aims at finding the best possible solution in parameter space according to pre-defined objective function and available data. Moreover, different optimization algorithms may perform better for simpler or larger ANN architectures. This suggest the importance of proper coupling of different optimization algorithms, ANN architectures and methods to avoid overfitting of real-world data – an issue that is also studied in details in the present paper. The study is performed for Annapolis River catchment, characterized by significant seasonal changes in runoff, rapid floods during winter and spring, moderately dry summers, severe winters with snowfall, snow melting, frequent freeze and thaw, and presence of river ice. The present paper shows that the elaborated noise injection method may prevent overfitting slightly better than the most popular early stopping approach. However, the implementation of noise injection to real-world problems is difficult and the final model performance depends significantly on a number of very technical details, what somehow limits its practical applicability. It is shown that optimized approximation algorithm does not improve the results obtained by older methods, possibly due to over-simplified criterion of stopping the algorithm. Extensive calculations reveal that Evolutionary Computation-based algorithm performs better for simpler ANN architectures, whereas classical gradient-based Levenberg–Marquardt algorithm is able to benefit from additional input variables, representing precipitation and snow cover from one more previous day, and from more complicated ANN architectures. This confirms that the curse of dimensionality has severe impact on the performance of Evolutionary Computing methods. [Copyright &y& Elsevier]

Abstract: In this paper Product-Units neural networks (PUNNs), which probably have never been used within the field of hydrology, are introduced and applied for catchment runoff forecasting in cold climate zones. This type of neural networks, a subclass of higher order neural networks uses product nodes with inputs raised to exponential weights in one layer and well-known summation nodes in another layer. The present paper empirically shows that PUNNs with unbounded weights are difficult to train and do not perform well for catchment runoff forecasting. However, a very good predictive performance may be achieved when the weights are bounded within [−1,1] interval. Several variants of optimization methods, mostly Differential Evolution-based algorithms, and a few approaches enabling good generalization capabilities of neural networks are compared in order to select the appropriate technique for PUNNs training. PUNNs with parameters bounded within [−1,1] interval are shown to outperform Multi-Layer Perceptron neural networks and HBV conceptual model for runoff forecasting case study at Annapolis River, Nova Scotia, Canada. Gradient-based Levenberg–Marquardt algorithm and Evolutionary Computation-based Differential Evolution with Global and Local Neighborhood method turn out to be the most successful among the tested training algorithms. Surprisingly, in the case of Product-Units neural networks with weights bounded within [−1,1] interval using noise injection or early stopping do not improve the results obtained when no method to avoid overfitting is used. [Copyright &y& Elsevier]

Abstract: This study presents the comparison of various evolutionary computation (EC) optimization techniques applied to train the noise-injected multi-layer perceptron neural networks used for estimation of longitudinal dispersion coefficient in rivers. The special attention is paid to recently developed variants of Differential Evolution (DE) algorithm. The most commonly used gradient-based optimization methods have two significant drawbacks: they cannot cope with non-differentiable problems and quickly converge to local optima. These problems can be avoided by the application of EC techniques. Although a great amount of various EC algorithms have been proposed in recent years, only some of them have been applied to neural network training – usually with no comparison to other methods. We restrict our comparison to the regression problem with limited data and noise injection technique used to avoid premature convergence and to improve robustness of the model. The optimization methods tested in the present paper are: Distributed DE with Explorative–Exploitative Population Families, Self-Adaptive DE, DE with Global and Local Neighbors, Grouping DE, JADE, Comprehensive Learning Particle Swarm Optimization, Efficient Population Utilization Strategy Particle Swarm Optimization and Covariance Matrix Adaptation – Evolution Strategy. [Copyright &y& Elsevier]

Summary: Although neural networks have been widely applied to various hydrological problems, including river flow forecasting, for at least 15years, they have usually been trained by means of gradient-based algorithms. Recently nature inspired Evolutionary Computation algorithms have rapidly developed as optimization methods able to cope not only with non-differentiable functions but also with a great number of local minima. Some of proposed Evolutionary Computation algorithms have been tested for neural networks training, but publications which compare their performance with gradient-based training methods are rare and present contradictory conclusions. The main goal of the present study is to verify the applicability of a number of recently developed Evolutionary Computation optimization methods, mostly from the Differential Evolution family, to multi-layer perceptron neural networks training for daily rainfall–runoff forecasting. In the present paper eight Evolutionary Computation methods, namely the first version of Differential Evolution (DE), Distributed DE with Explorative–Exploitative Population Families, Self-Adaptive DE, DE with Global and Local Neighbors, Grouping DE, JADE, Comprehensive Learning Particle Swarm Optimization and Efficient Population Utilization Strategy Particle Swarm Optimization are tested against the Levenberg–Marquardt algorithm – probably the most efficient in terms of speed and success rate among gradient-based methods. The Annapolis River catchment was selected as the area of this study due to its specific climatic conditions, characterized by significant seasonal changes in runoff, rapid floods, dry summers, severe winters with snowfall, snow melting, frequent freeze and thaw, and presence of river ice – conditions which make flow forecasting more troublesome. The overall performance of the Levenberg–Marquardt algorithm and the DE with Global and Local Neighbors method for neural networks training turns out to be superior to other Evolutionary Computation-based algorithms. The Levenberg–Marquardt optimization must be considered as the most efficient one due to its speed. Its drawback due to possible sticking in poor local optimum can be overcome by applying a multi-start approach. [Copyright &y& Elsevier]

• Depending on stream temperature model, different warming of streamwater is projected. • Projection from particular model occasionally differ from the majority of models. • A number of stream temperature models should be used together for future climates. • Stream temperature in analyzed streams is to be warmed by 2–3.5 °C by 2100 (RCP 8.5). • In specific streams marginal cooling in projected in some months until 2050. In the majority of studies aiming at stream temperature warming due to climate change just a single water temperature model is used. Choosing a single model may highly impact the conclusions from the study. In this paper four relatively different empirical or semi-empirical models: perceptron neural networks, product unit networks, extended logistic regression and air2stream were applied to project the impact of climate change on water temperature in rivers located in temperate climatic zones of the USA and Poland. The models were driven by daily air temperature and streamflow projected by the rainfall-runoff model. In the first step, the models were calibrated and validated. Then the projections of water temperature were derived for the historical periods and two future periods taking into account: (a) climate simulations from the CORDEX initiative (NA-CORDEX and EURO-CORDEX), (b) the GR4J rainfall-runoff model and (c) different water temperature models. The obtained results indicate that due to global warming, the stream temperatures are expected to increase by about 1–2 °C for 2021–2050 and by 2–3 °C for 2071–2100 periods. These changes are not uniformly distributed throughout the year. The largest warming in the USA is found in the summertime, in Poland – in spring and autumn. For some months the discrepancies in the projected stream temperature between various stream temperature models are large. Product unit neural network, logistic regression-based model or air2stream occasionally led to projections that differ from those obtained by the majority of models even by 2 °C. We strongly recommend using at least a few stream temperature models for analysing the impact of climate change on water temperatures or the fate of the aquatic ecosystem. [ABSTRACT FROM AUTHOR]

• Product unit neural networks (PUNN) are implemented for stream temperature modelling. • Stream temperature modelling is based on atmospheric and hydrological interactions. • Deep learning-based dropout variant is proposed for shallow PUNN. • Impact of various dropout variants is tested for PUNN. • PUNN with proposed dropout outperforms data-based and semi-physical models. For about two decades neural networks are widely used for river temperature modelling. However, in recent years one has to distinguish between the "classical" shallow neural networks, and deep learning networks. The applicability of rapidly developing deep learning networks to stream water temperature modelling may be limited, but some methods developed for deep learning, if properly re-considered, may efficiently improve performance of shallow networks. Dropout is widely considered the method that allows deep learning networks to avoid overfitting to training data, facilitating its implementations to versatile problems. Recently the successful applicability of dropout for river temperature modelling by means of shallow multilayer perceptron neural networks has been introduced. In the present study we propose to use dropout solely for input neurons of product unit neural networks for the purpose of stream temperature modelling. We perform tests on data collected from six catchments located in temperate climate zones on two continents in various orographic conditions. We show that the average performance of product unit neural networks trained with input dropout is better than the average performance of product units without dropout, product units with dropout applied to every layer of the networks, multilayer perceptron neural networks with or without dropout, and the semi-physical air2stream model. The advantage of product unit neural networks with input dropout is statistically significant on hilly or mountainous catchments; the performance on flat ones is similar to the performances obtained from competitive models. [ABSTRACT FROM AUTHOR]

• 12 algorithms were implemented for the air2water model. • Impact of the number of function calls was evaluated. • The HARD-DE algorithm performed the best. The air2water model is a widely used tool for the forecasting of lake water temperatures and projection of climate change. It has been applied in thousands of lakes worldwide. However, it employed the Particle Swarm Optimization (PSO) as the calibration method, which was proposed more than 20 years ago. In this study, 12 advanced optimization algorithms proposed during the recent few years were implemented to calibrate and validate the air2water model. Daily observed water temperature in temperate lakes (22 lowland lakes in Poland) were used to evaluate the model performance. Each optimizer was run 30 times with 3 different settings of the maximum number of function calls, 5000, 20,000, and 100,000. We have found that the PSO method used in the original air2water model performs relatively poor compared with most of the recent algorithms, and the majority of recently proposed algorithms reach the similar best solutions on many lakes when the highest number of function calls is allowed. However, only the HARD-DE (hierarchical archive-based mutation strategy with depth information of differential evolution) algorithm is never outperformed by any competitor, irrespective which lake and maximum number of function calls is considered. As a result, we highly recommend using HARD-DE as the calibration method for the air2water model in the future studies. [ABSTRACT FROM AUTHOR]

Although deep learning applicability in various fields of earth sciences is rapidly increasing, shallow multilayer-perceptron neural networks remain widely used for regression problems. Despite many clear distinctions between deep and shallow neural networks, some techniques developed for deep learning may help improve shallow models. Dropout, a simple approach to avoid overfitting by randomly skipping some nodes in a net during each training iteration, is among methodological features that made deep learning networks successful. In this study we give a review of dropout methods and empirically show that, when used together with early-stopping, dropout and its variant dropconnect could improve performance of shallow multi-layer perceptron neural networks. Shallow neural networks are applied to streamwater temperature modelling problems in six catchments, based on air temperature, river discharge and declination of the Sun. We found that when training of a particular neural network architecture that includes at least a few hidden nodes is repeated many times, dropout reduces the number of models that perform poorly on testing data, and hence improves the mean performance. If the number of inputs or hidden nodes is very low, dropout only disturbs training. However, nodes need to be dropped out with a much lower probability than in the case of deep neural networks (about 1%, instead of 10–50% for deep learning), due to a much smaller number of nodes in the network. Larger probabilities of dropping out nodes hinder convergence of the training algorithm and lead to poor results for both calibration and testing data. Dropconnect turned out to be slightly more effective than dropout. [ABSTRACT FROM AUTHOR]