Search

• New logistic regression model is proposed for stream temperature simulations. • Air temperature, flow and declination of the Sun are used as input variables. • Proposed model is simple and may easily be applied in practice. • Proposed model is tested on six streams located in various morphological conditions. • Proposed model outperforms other logistic regression models and semi-physical model. Among various stream temperature models those based on nonlinear regression frequently attract attention due to their simplicity and small number of required variables. Among such approaches the logistic regression model developed twenty years ago for weekly data is still widely used in various scientific studies that require quick and simple calculation of stream water temperature. The model has been modified a number of times in recent years to capture the relationship between daily stream water temperatures, air temperatures and discharge. In this study, we propose further modifications of the logistic regression model that do not require any additional variables that may be hard to measure. The proposed models capture the relationship between the stream temperature and the declination of the Sun, the air temperature and the discharge from a number of recent observations. The proposed approaches are tested on six rivers located in diverse orographic conditions of temperate climate zones of Europe and USA. Although the proposed models remain very simple, their performances are competitive against the performances of more advanced semi-physical or data-driven models. [ABSTRACT FROM AUTHOR]

Artificial neural networks (ANNs) become widely used for runoff forecasting in numerous studies. Usually classical gradient-based methods are applied in ANN training and a single ANN model is used. To improve the modelling performance, in some papers ensemble aggregation approaches are used whilst in others, novel training methods are proposed. In this study, the usefulness of both concepts is analysed. First, the applicability of a large number of population-based metaheuristics to ANN training for runoff forecasting is tested on data collected from four catchments, namely upper Annapolis (Nova Scotia, Canada), Biala Tarnowska (Poland), upper Allier (France) and Axe Creek (Victoria, Australia). Then, the importance of the search for novel training methods is compared with the importance of the use of a very simple ANN ensemble aggregation approach. It is shown that although some metaheuristics may slightly outperform the classical gradient-based Levenberg-Marquardt algorithm for a specific catchment, none performs better for the majority of the tested ones. One may also point out a few metaheuristics that do not suit ANN training at all. On the other hand, application of even the simplest ensemble aggregation approach clearly improves the results when the ensemble members are trained by any suitable algorithms.EDITOR D. Koutsoyiannis; ASSOCIATE EDITOR E. Toth [ABSTRACT FROM AUTHOR]

Summary A number of methods have been proposed for the prediction of streamwater temperature based on various meteorological and hydrological variables. The present study shows a comparison of few types of data-driven neural networks (multi-layer perceptron, product-units, adaptive-network-based fuzzy inference systems and wavelet neural networks) and nearest neighbour approach for short time streamwater temperature predictions in two natural catchments (mountainous and lowland) located in temperate climate zone, with snowy winters and hot summers. To allow wide applicability of such models, autoregressive inputs are not used and only easily available measurements are considered. Each neural network type is calibrated independently 100 times and the mean, median and standard deviation of the results are used for the comparison. Finally, the ensemble aggregation approach is tested. The results show that simple and popular multi-layer perceptron neural networks are in most cases not outperformed by more complex and advanced models. The choice of neural network is dependent on the way the models are compared. This may be a warning for anyone who wish to promote own models, that their superiority should be verified in different ways. The best results are obtained when mean, maximum and minimum daily air temperatures from the previous days are used as inputs, together with the current runoff and declination of the Sun from two recent days. The ensemble aggregation approach allows reducing the mean square error up to several percent, depending on the case, and noticeably diminishes differences in modelling performance obtained by various neural network types. [ABSTRACT FROM AUTHOR]

Highlights: [•] Differential Evolution algorithms applied to ANN training suffer from stagnation. [•] The lack of difference vectors of small magnitude is noted during ANN training by Differential Evolution methods. [•] In case of benchmark problems the lack of difference vectors of small magnitude is only occasionally observed. [•] DEGL algorithm outperforms other Differential Evolution variants for ANN training. [•] Best algorithms found for benchmark problems do not perform well for ANN training. [ABSTRACT FROM AUTHOR]

Abstract: Nature-inspired metaheuristics found various applications in different fields of science, including the problem of artificial neural networks (ANN) training. However, very versatile opinions regarding the performance of metaheuristics applied to ANN training may be found in the literature. Both nature-inspired metaheuristics and ANNs are widely applied to various geophysical and environmental problems. Among them the water temperature forecasting in a natural river, especially in colder climate zones where the seasonality plays important role, is of great importance, as water temperature has strong impact on aquatic life and chemistry. As the impact of possible future climate change on water temperature is not trivial, models are needed to allow projection of streamwater temperature based on simple hydro-meteorological variables. In this paper the detailed comparison of the performance of nature-inspired optimization methods and Levenberg–Marquardt (LM) algorithm in ANNs training is performed, based on the case study of water temperature forecasting in a natural stream, namely Biala Tarnowska river in southern Poland. Over 50 variants of 22 various metaheuristics, including a large number of Differential Evolution, as well as some Particle Swarm Optimization, Evolution Strategies, multialgorithms and Direct Search methods are compared with LM algorithm on ANN training for the described case study. The impact of population size and some control parameters of particular metaheuristics on the ANN training performance are verified. It is found that despite widely claimed large improvement in nature-inspired methods during last years, the vast majority of them are still outperformed by LM algorithm on the selected problem. The only methods that, based on this case study, seem competitive to LM algorithm in terms of the final performance (but not speed) are Differential Evolution algorithms that benefit from the concept of Global and Local neighborhood-based mutation operators. The streamwater forecasting performance of the neural networks is adequate, the major prediction errors are related to the river freezing and melting processes that occur during winter in the mountainous catchment under study. [Copyright &y& Elsevier]

Summary: Artificial neural networks (ANNs) becomes very popular tool in hydrology, especially in rainfall–runoff modelling. However, a number of issues should be addressed to apply this technique to a particular problem in an efficient way, including selection of network type, its architecture, proper optimization algorithm and a method to deal with overfitting of the data. The present paper addresses the last, rarely considered issue, namely comparison of methods to prevent multi-layer perceptron neural networks from overfitting of the training data in the case of daily catchment runoff modelling. Among a number of methods to avoid overfitting the early stopping, the noise injection and the weight decay have been known for about two decades, however only the first one is frequently applied in practice. Recently a new methodology called optimized approximation algorithm has been proposed in the literature. Overfitting of the training data leads to deterioration of generalization properties of the model and results in its untrustworthy performance when applied to novel measurements. Hence the purpose of the methods to avoid overfitting is somehow contradictory to the goal of optimization algorithms, which aims at finding the best possible solution in parameter space according to pre-defined objective function and available data. Moreover, different optimization algorithms may perform better for simpler or larger ANN architectures. This suggest the importance of proper coupling of different optimization algorithms, ANN architectures and methods to avoid overfitting of real-world data – an issue that is also studied in details in the present paper. The study is performed for Annapolis River catchment, characterized by significant seasonal changes in runoff, rapid floods during winter and spring, moderately dry summers, severe winters with snowfall, snow melting, frequent freeze and thaw, and presence of river ice. The present paper shows that the elaborated noise injection method may prevent overfitting slightly better than the most popular early stopping approach. However, the implementation of noise injection to real-world problems is difficult and the final model performance depends significantly on a number of very technical details, what somehow limits its practical applicability. It is shown that optimized approximation algorithm does not improve the results obtained by older methods, possibly due to over-simplified criterion of stopping the algorithm. Extensive calculations reveal that Evolutionary Computation-based algorithm performs better for simpler ANN architectures, whereas classical gradient-based Levenberg–Marquardt algorithm is able to benefit from additional input variables, representing precipitation and snow cover from one more previous day, and from more complicated ANN architectures. This confirms that the curse of dimensionality has severe impact on the performance of Evolutionary Computing methods. [Copyright &y& Elsevier]

Abstract: This study presents the comparison of various evolutionary computation (EC) optimization techniques applied to train the noise-injected multi-layer perceptron neural networks used for estimation of longitudinal dispersion coefficient in rivers. The special attention is paid to recently developed variants of Differential Evolution (DE) algorithm. The most commonly used gradient-based optimization methods have two significant drawbacks: they cannot cope with non-differentiable problems and quickly converge to local optima. These problems can be avoided by the application of EC techniques. Although a great amount of various EC algorithms have been proposed in recent years, only some of them have been applied to neural network training – usually with no comparison to other methods. We restrict our comparison to the regression problem with limited data and noise injection technique used to avoid premature convergence and to improve robustness of the model. The optimization methods tested in the present paper are: Distributed DE with Explorative–Exploitative Population Families, Self-Adaptive DE, DE with Global and Local Neighbors, Grouping DE, JADE, Comprehensive Learning Particle Swarm Optimization, Efficient Population Utilization Strategy Particle Swarm Optimization and Covariance Matrix Adaptation – Evolution Strategy. [Copyright &y& Elsevier]

Summary: Although neural networks have been widely applied to various hydrological problems, including river flow forecasting, for at least 15years, they have usually been trained by means of gradient-based algorithms. Recently nature inspired Evolutionary Computation algorithms have rapidly developed as optimization methods able to cope not only with non-differentiable functions but also with a great number of local minima. Some of proposed Evolutionary Computation algorithms have been tested for neural networks training, but publications which compare their performance with gradient-based training methods are rare and present contradictory conclusions. The main goal of the present study is to verify the applicability of a number of recently developed Evolutionary Computation optimization methods, mostly from the Differential Evolution family, to multi-layer perceptron neural networks training for daily rainfall–runoff forecasting. In the present paper eight Evolutionary Computation methods, namely the first version of Differential Evolution (DE), Distributed DE with Explorative–Exploitative Population Families, Self-Adaptive DE, DE with Global and Local Neighbors, Grouping DE, JADE, Comprehensive Learning Particle Swarm Optimization and Efficient Population Utilization Strategy Particle Swarm Optimization are tested against the Levenberg–Marquardt algorithm – probably the most efficient in terms of speed and success rate among gradient-based methods. The Annapolis River catchment was selected as the area of this study due to its specific climatic conditions, characterized by significant seasonal changes in runoff, rapid floods, dry summers, severe winters with snowfall, snow melting, frequent freeze and thaw, and presence of river ice – conditions which make flow forecasting more troublesome. The overall performance of the Levenberg–Marquardt algorithm and the DE with Global and Local Neighbors method for neural networks training turns out to be superior to other Evolutionary Computation-based algorithms. The Levenberg–Marquardt optimization must be considered as the most efficient one due to its speed. Its drawback due to possible sticking in poor local optimum can be overcome by applying a multi-start approach. [Copyright &y& Elsevier]

• Depending on stream temperature model, different warming of streamwater is projected. • Projection from particular model occasionally differ from the majority of models. • A number of stream temperature models should be used together for future climates. • Stream temperature in analyzed streams is to be warmed by 2–3.5 °C by 2100 (RCP 8.5). • In specific streams marginal cooling in projected in some months until 2050. In the majority of studies aiming at stream temperature warming due to climate change just a single water temperature model is used. Choosing a single model may highly impact the conclusions from the study. In this paper four relatively different empirical or semi-empirical models: perceptron neural networks, product unit networks, extended logistic regression and air2stream were applied to project the impact of climate change on water temperature in rivers located in temperate climatic zones of the USA and Poland. The models were driven by daily air temperature and streamflow projected by the rainfall-runoff model. In the first step, the models were calibrated and validated. Then the projections of water temperature were derived for the historical periods and two future periods taking into account: (a) climate simulations from the CORDEX initiative (NA-CORDEX and EURO-CORDEX), (b) the GR4J rainfall-runoff model and (c) different water temperature models. The obtained results indicate that due to global warming, the stream temperatures are expected to increase by about 1–2 °C for 2021–2050 and by 2–3 °C for 2071–2100 periods. These changes are not uniformly distributed throughout the year. The largest warming in the USA is found in the summertime, in Poland – in spring and autumn. For some months the discrepancies in the projected stream temperature between various stream temperature models are large. Product unit neural network, logistic regression-based model or air2stream occasionally led to projections that differ from those obtained by the majority of models even by 2 °C. We strongly recommend using at least a few stream temperature models for analysing the impact of climate change on water temperatures or the fate of the aquatic ecosystem. [ABSTRACT FROM AUTHOR]

• Product unit neural networks (PUNN) are implemented for stream temperature modelling. • Stream temperature modelling is based on atmospheric and hydrological interactions. • Deep learning-based dropout variant is proposed for shallow PUNN. • Impact of various dropout variants is tested for PUNN. • PUNN with proposed dropout outperforms data-based and semi-physical models. For about two decades neural networks are widely used for river temperature modelling. However, in recent years one has to distinguish between the "classical" shallow neural networks, and deep learning networks. The applicability of rapidly developing deep learning networks to stream water temperature modelling may be limited, but some methods developed for deep learning, if properly re-considered, may efficiently improve performance of shallow networks. Dropout is widely considered the method that allows deep learning networks to avoid overfitting to training data, facilitating its implementations to versatile problems. Recently the successful applicability of dropout for river temperature modelling by means of shallow multilayer perceptron neural networks has been introduced. In the present study we propose to use dropout solely for input neurons of product unit neural networks for the purpose of stream temperature modelling. We perform tests on data collected from six catchments located in temperate climate zones on two continents in various orographic conditions. We show that the average performance of product unit neural networks trained with input dropout is better than the average performance of product units without dropout, product units with dropout applied to every layer of the networks, multilayer perceptron neural networks with or without dropout, and the semi-physical air2stream model. The advantage of product unit neural networks with input dropout is statistically significant on hilly or mountainous catchments; the performance on flat ones is similar to the performances obtained from competitive models. [ABSTRACT FROM AUTHOR]