Search

We introduce a new diagnosis tool that is well suited to analyzing simulation results over large samples of watersheds. It consists of a modification of the classical Taylor diagram to simultaneously visualize several error components (based on bias, standard deviation or squared errors) that are commonly used in efficiency criteria (such as the Nash–Sutcliffe efficiency (NSE) or the Kling–Gupta efficiency (KGE)) to evaluate hydrological model performance. We propose a methodological framework that explicitly links the graphical and numerical evaluation approaches, and show how they can be usefully combined to visually interpret numerical experiments conducted on large datasets. The approach is illustrated using results obtained by testing two rainfall-runoff models on a sample of 2050 watersheds from 8 countries and calibrated with two alternative objective functions (NSE and KGE). The assessment tool clearly highlights well-documented problems related to the use of the NSE for the calibration of rainfall-runoff models, which arise due to interactions between the ratio of simulated to observed standard deviations and the correlation coefficient. We also illustrate the negative impacts of classical mathematical transformations (square root) applied to streamflow when employing NSE and KGE as metrics for model calibration.

Process-based modelling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. In this Perspective, we explore differentiable modelling as a pathway to dissolve the perceived barrier between process-based modelling and machine learning in the geosciences and demonstrate its potential with examples from hydrological modelling. ‘Differentiable’ refers to accurately and efficiently calculating gradients with respect to model variables or parameters, enabling the discovery of high-dimensional unknown relationships. Differentiable modelling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints. Differentiable modelling approaches are primed to enable geoscientists to ask questions, test hypotheses, and discover unrecognized physical relationships. Future work should address computational challenges, reduce uncertainty, and verify the physical significance of outputs.

This study leverages a Random Forest model to predict flood hazard in Arizona, New Mexico, Colorado, and Utah, focusing on enhancing sustainability in flood management. Utilizing the National Flood Hazard Layer (NFHL), an intricate flood map of Arizona was generated, with the Random Forest Classification algorithm assessing flood hazard for each grid cell. Weather variable predictions from TerraClimate were integrated with NFHL classifications and Digital Elevation Model (DEM) analyses, providing a comprehensive understanding of flood dynamics. The research highlights the model's capability to predict flood hazard in areas lacking NFHL classifications, thereby supporting sustainable flood management by elucidating weather's influence on flood hazard. This approach aligns with sustainable development goals by aiding in resilient infrastructure design and informed urban planning, reducing the impact of floods on communities. Despite recognizing constraints such as input data precision and the model's potential limitations in capturing complex variable interactions, the methodology offers a robust framework for flood hazard evaluation in other regions. Integrating diverse data sources, this study presents a valuable tool for decision-makers, supporting sustainable practices, and enhancing the resilience of vulnerable regions against flood hazards. This integrated approach underscores the potential of advanced modeling techniques in promoting sustainability in environmental hazard management. [ABSTRACT FROM AUTHOR]

We investigate the applicability of machine learning technologies to the development of parsimonious, interpretable, catchment‐scale hydrologic models using directed‐graph architectures based on the mass‐conserving perceptron (MCP) as the fundamental computational unit. Here, we focus on architectural complexity (depth) at a single location, rather than universal applicability (breadth) across large samples of catchments. The goal is to discover a minimal representation (numbers of cell‐states and flow paths) that represents the dominant processes that can explain the input‐state‐output behaviors of a given catchment, with particular emphasis given to simulating the full range (high, medium, and low) of flow dynamics. We find that a "HyMod Like" architecture with three cell‐states and two major flow pathways achieves such a representation at our study location, but that the additional incorporation of an input‐bypass mechanism significantly improves the timing and shape of the hydrograph, while the inclusion of bi‐directional groundwater mass exchanges significantly enhances the simulation of baseflow. Overall, our results demonstrate the importance of using multiple diagnostic metrics for model evaluation, while highlighting the need for properly selecting and designing the training metrics based on information‐theoretic foundations that are better suited to extracting information across the full range of flow dynamics. This study sets the stage for interpretable regional‐scale MCP‐based hydrological modeling (using large sample data) by using neural architecture search to determine appropriate minimal representations for catchments in different hydroclimatic regimes. Plain Language Summary: We show that conventional machine learning technologies can be used to develop parsimonious, interpretable, catchment‐scale hydrologic models using the mass‐conserving perceptron (MCP) as a fundamental computational unit. Using data from the Leaf River Basin, we test a variety of minimal, dominant process, representations that can explain the input‐state‐output dynamics of the catchment. Our results demonstrate the importance of using multiple diagnostic metrics for evaluation and comparison of different model architectures, and highlight the importance of choosing (or designing) objective functions for model training that are properly suited to the task of extracting information across the full range of flow dynamics. This depth‐focus study sets the stage for interpretable regional‐scale MCP‐based hydrological modeling (using large sample data) by using neural architecture search to determine appropriate minimal representations for catchments in different hydroclimatic regimes. Key Points: We utilize mass‐conserving perceptron (MCP) directed‐graph architectures to develop concise, interpretable catchment‐scale hydrologic modelsWe focus on model complexity (depth) at a single location, rather than universal applicability (breadth) across large samples of catchmentsThis study set the stage for interpretable MCP‐based modeling to find minimal representations in different hydroclimatic regimes [ABSTRACT FROM AUTHOR]

Using information-theoretic quantities in practical applications with continuous data is often hindered by the fact that probability density functions need to be estimated in higher dimensions, which can become unreliable or even computationally unfeasible. To make these useful quantities more accessible, alternative approaches such as binned frequencies using histograms and k-nearest neighbors (k-NN) have been proposed. However, a systematic comparison of the applicability of these methods has been lacking. We wish to fill this gap by comparing kernel-density-based estimation (KDE) with these two alternatives in carefully designed synthetic test cases. Specifically, we wish to estimate the information-theoretic quantities: entropy, Kullback–Leibler divergence, and mutual information, from sample data. As a reference, the results are compared to closed-form solutions or numerical integrals. We generate samples from distributions of various shapes in dimensions ranging from one to ten. We evaluate the estimators' performance as a function of sample size, distribution characteristics, and chosen hyperparameters. We further compare the required computation time and specific implementation challenges. Notably, k-NN estimation tends to outperform other methods, considering algorithmic implementation, computational efficiency, and estimation accuracy, especially with sufficient data. This study provides valuable insights into the strengths and limitations of the different estimation methods for information-theoretic quantities. It also highlights the significance of considering the characteristics of the data, as well as the targeted information-theoretic quantity when selecting an appropriate estimation technique. These findings will assist scientists and practitioners in choosing the most suitable method, considering their specific application and available data. We have collected the compared estimation methods in a ready-to-use open-source Python 3 toolbox and, thereby, hope to promote the use of information-theoretic quantities by researchers and practitioners to evaluate the information in data and models in various disciplines. [ABSTRACT FROM AUTHOR]

Several studies have demonstrated the ability of long short-term memory (LSTM) machine-learning-based modeling to outperform traditional spatially lumped process-based modeling approaches for streamflow prediction. However, due mainly to the structural complexity of the LSTM network (which includes gating operations and sequential processing of the data), difficulties can arise when interpreting the internal processes and weights in the model. Here, we propose and test a modification of LSTM architecture that is calibrated in a manner that is analogous to a hydrological system. Our architecture, called "HydroLSTM", simulates the sequential updating of the Markovian storage while the gating operation has access to historical information. Specifically, we modify how data are fed to the new representation to facilitate simultaneous access to past lagged inputs and consolidated information, which explicitly acknowledges the importance of trends and patterns in the data. We compare the performance of the HydroLSTM and LSTM architectures using data from 10 hydro-climatically varied catchments. We further examine how the new architecture exploits the information in lagged inputs, for 588 catchments across the USA. The HydroLSTM-based models require fewer cell states to obtain similar performance to their LSTM-based counterparts. Further, the weight patterns associated with lagged input variables are interpretable and consistent with regional hydroclimatic characteristics (snowmelt-dominated, recent rainfall-dominated, and historical rainfall-dominated). These findings illustrate how the hydrological interpretability of LSTM-based models can be enhanced by appropriate architectural modifications that are physically and conceptually consistent with our understanding of the system. [ABSTRACT FROM AUTHOR]

Weather radar systems provide detailed information on spatial rainfall patterns known to play a significant role in runoff generation processes. In the current study, we present an innovative approach to exploit spatial rainfall information of air mass thunderstorms and link it with a watershed hydrological model. Observed radar data are decomposed into sets of rain cells conceptualized as circular Gaussian elements and the associated rain cell parameters, namely, location, maximal intensity and decay factor, are input into a hydrological model. Rain cells were retrieved from radar data for several thunderstorms over southern Arizona. Spatial characteristics of the resulting rain fields were evaluated using data from a dense rain gauge network. For an extreme case study in a semi-arid watershed, rain cells were derived and fed as input into a hydrological model to compute runoff response. A major factor in this event was found to be a single intense rain cell (out of the five cells decomposed from the storm). The path of this cell near watershed tributaries and toward the outlet enhanced generation of high flow. Furthermore, sensitivity analysis to cell characteristics indicated that peak discharge could be a factor of two higher if the cell was initiated just a few kilometers aside. © 2005 Elsevier Ltd. All rights reserved.

Hydrological models generally contain parameters that cannot be measured directly, but can only be meaningfully inferred by calibration against a historical record of input-output data. While considerable progress has been made in the development and application of automatic procedures for model calibration, such methods have received criticism for their lack of rigor in treating uncertainty in the parameter estimates. In this paper, we apply the recently developed Shuffled Complex Evolution Metropolis algorithm (SCEM-UA) to stochastic calibration of the parameters in the Sacramento Soil Moisture Accounting model (SAC-SMA) model using historical data from the Leaf River in Mississippi. The SCEM-UA algorithm is a Markov Chain Monte Carlo sampler that provides an estimate of the most likely parameter set and underlying posterior distribution within a single optimization run. In particular, we explore the relationship between the length and variability of the streamflow data and the Bayesian uncertainty associated with the SAC-SMA model parameters and compare SCEM-UA derived parameter values with those obtained using deterministic SCE-UA calibrations. Most significantly, for the Leaf River catchments under study our results demonstrate that most of the 13 SAC-SMA parameters are well identified by calibration to daily streamflow data suggesting that this data contains more information than has previously been reported in the literature. © 2006 Elsevier B.V. All rights reserved.

This study compares mean areal precipitation (MAP) estimates derived from three sources: an operational rain gauge network (MAPG), a radar/ gauge multisensor product (MAPX), and the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) satellite-based system (MAPS) for the time period from March 2000 to November 2003. The study area includes seven operational basins of varying size and location in the southeastern United States. The analysis indicates that agreements between the datasets vary considerably from basin to basin and also temporally within the basins. The analysis also includes evaluation of MAPS in comparison with MAPG for use in flow forecasting with a lumped hydrologic model [Sacramento Soil Moisture Accounting Model (SAC-SMA)]. The latter evaluation investigates two different parameter sets, the first obtained using manual calibration on historical MAPG, and the second obtained using automatic calibration on both MAPS and MAPG, but over a shorter time period (23 months). Results indicate that the overall performance of the model simulations using MAPS depends on both the bias in the precipitation estimates and the size of the basins, with poorer performance in basins of smaller size (large bias between MAPG and MAPS) and better performance in larger basins (less bias between MAPG and MAPS). When using MAPS, calibration of the parameters significantly improved the model performance. © 2005 American Meteorological Society.

A reliable prediction of hydrologic models, among other things, requires a set of plausible parameters that correspond with physiographic properties of the basin. This study proposes a parameter estimation approach, which is based on extracting, through hydrograph diagnoses, information in the form of indices that carry intrinsic properties of a basin. This concept is demonstrated by introducing two indices that describe the shape of a streamflow hydrograph in an integrated manner. Nineteen mid-size (223-4790 km2) perennial headwater basins with a long record of streamflow data were selected to evaluate the ability of these indices to capture basin response characteristics. An examination of the utility of the proposed indices in parameter estimation is conducted for a five-parameter hydrologic model using data from the Leaf River, located in Fort Collins, Mississippi. It is shown that constraining the parameter estimation by selecting only those parameters that result in model output which maintains the indices as found in the historical data can improve the reliability of model predictions. These improvements were manifested in (a) improvement of the prediction of low and high flow, (b) improvement of the overall total biases, and (c) maintenance of the hydrograph's shape for both long-term and short-term predictions. Copyright © 2005 John Wiley & Sons, Ltd.

This paper presents a parameter estimation approach based on hydrograph descriptors that capture dominant streamflow characteristics at three timescales (monthly, yearly, and record extent). The scheme, entitled hydrograph descriptors multitemporal sensitivity analyses (HYDMUS), yields an ensemble of model simulations generated from a reduced parameter space, based on a set of streamflow descriptors that emphasize the timescale dynamics of streamflow record. In this procedure the posterior distributions of model parameters derived at coarser timescales are used to sample model parameters for the next finer timescale. The procedure was used to estimate the parameters of the Sacramento soil moisture accounting model (SAC-SMA) for the Leaf River, Mississippi. The results indicated that in addition to a significant reduction in the range of parameter uncertainty, HYDMUS improved parameter identifiability for all 13 of the model parameters. The performance of the procedure was compared to four previous calibration studies on the same watershed. Although our application of HYDMUS did not explicitly consider the error at each simulation time step during the calibration process, the model performance was, in some important respects, found to be better than in previous deterministic studies. Copyright 2005 by the American Geophysical Union.

Two elementary issues in contemporary Earth system science and engineering are (1) the specification of model parameter values which characterize a system and (2) the estimation of state variables which express the system dynamic. This paper explores a novel sequential hydrologic data assimilation approach for estimating model parameters and state variables using particle filters (PFs). PFs have their origin in Bayesian estimation. Methods for batch calibration, despite major recent advances, appear to lack the flexibility required to treat uncertainties in the current system as new information is received. Methods based on sequential Bayesian estimation seem better able to take advantage of the temporal organization and structure of information, so that better compliance of the model output with observations can be achieved. Such methods provide platforms for improved uncertainty assessment and estimation of hydrologic model components, by providing more complete and accurate representations of the forecast and analysis probability distributions. This paper introduces particle filtering as a sequential Bayesian filtering having features that represent the full probability distribution of predictive uncertainties. Particle filters have, so far, generally been used to recursively estimate the posterior distribution of the model state; this paper investigates their applicability to the approximation of the posterior distribution of parameters. The capability and usefulness of particle filters for adaptive inference of the joint posterior distribution of the parameters and state variables are illustrated via two case studies using a parsimonious conceptual hydrologic model. Copyright 2005 by the American Geophysical Union.

Hydrologic models are twofold: models for understanding physical processes and models for prediction. This study addresses the latter, which modelers use to predict, for example, streamflow at some future time given knowledge of the current state of the system and model parameters. In this respect, good estimates of the parameters and state variables are needed to enable the model to generate accurate forecasts. In this paper, a dual state-parameter estimation approach is presented based on the Ensemble Kalman Filter (EnKF) for sequential estimation of both parameters and state variables of a hydrologic model. A systematic approach for identification of the perturbation factors used for ensemble generation and for selection of ensemble size is discussed. The dual EnKF methodology introduces a number of novel features: (1) both model states and parameters can be estimated simultaneously; (2) the algorithm is recursive and therefore does not require storage of all past information, as is the case in the batch calibration procedures; and (3) the various sources of uncertainties can be properly addressed, including input, output, and parameter uncertainties. The applicability and usefulness of the dual EnKF approach for ensemble streamflow forecasting is demonstrated using a conceptual rainfall-runoff model. © 2004 Elsevier Ltd. All rights reserved.