Search

Hydraulic geometry parameters describing river hydrogeomorphic relationships are critical for determining a channel's capacity to convey water and sediment which is important for flood forecasting. Although well‐established, power‐law hydraulic geometry curves have been widely used to understand riverine systems and mapping flooding inundation worldwide for the past 70 years, we have become increasingly aware of their limitations. In the present study, we have moved beyond these traditional power‐law relationships, testing the ability of machine‐learning models to provide improved predictions of river width and depth. For this work, we have used an unprecedentedly large river measurement data set (HYDRoSWOT) as well as a suite of watershed predictor data to develop novel data‐driven approaches to better estimate river geometries over the contiguous United States (CONUS). Our Random Forest, XGBoost, and neural network models out‐performed the traditional, regionalized power law‐based hydraulic geometry equations for both width and depth, providing R‐squared values of as high as 0.75 for width and as high as 0.67 for depth, compared with R‐squared values of 0.45 for width and 0.18 for depth from the regional hydraulic geometry equations. Our results also show diverse performance outcomes across stream orders and geographical regions for the different machine‐learning models, demonstrating the value of using multi‐model approaches to maximize the predictability of river geometry. The developed models have been used to create the newly publicly available STREAM‐geo data set, which provides river width, depth, width/depth ratio, and river and stream surface area (%RSSA) for nearly 2.7 million NHDPlus stream reaches across the contiguous US. Plain Language Summary: Scientists and river managers use measurements of river geometry such as width and depth to forecast floods and understand river behavior. However, the methods used to estimate river geometry that have been used for decades are imprecise and thus lead to poor predictions of river discharge dynamics. Here, we've used new machine learning‐based modeling approaches to provide better predictions of river width and depth. We tested different machine‐learning models, which were developed based on the HYDRoSWOT set of measurements of rivers across the U.S. These new models all provide better estimates of river width and depth than the old methods. Our research can help us to provide better estimates of flood dynamics and improve our understanding of rivers across the U.S. Key Points: Machine Learning models outperform regional (physiographic) hydraulic geometry equations for predicting stream width and depthModel performance varies by stream orders and geographical regions, demonstrating the utility of multi‐model machine‐learning approachesThe STREAM‐geo data set provides predictions of river width, depth, width‐to‐depth ratio, and river area for the NHDPlus stream reaches [ABSTRACT FROM AUTHOR]

Accurate hydrologic modeling is vital to characterizing how the terrestrial water cycle responds to climate change. Pure deep learning (DL) models have been shown to outperform process-based ones while remaining difficult to interpret. More recently, differentiable physics-informed machine learning models with a physical backbone can systematically integrate physical equations and DL, predicting untrained variables and processes with high performance. However, it is unclear if such models are competitive for global-scale applications with a simple backbone. Therefore, we use – for the first time at this scale – differentiable hydrologic models (full name δ HBV-globe1.0-hydroDL, shortened to δ HBV here) to simulate the rainfall–runoff processes for 3753 basins around the world. Moreover, we compare the δ HBV models to a purely data-driven long short-term memory (LSTM) model to examine their strengths and limitations. Both LSTM and the δ HBV models provide competitive daily hydrologic simulation capabilities in global basins, with median Kling–Gupta efficiency values close to or higher than 0.7 (and 0.78 with LSTM for a subset of 1675 basins with long-term discharge records), significantly outperforming traditional models. Moreover, regionalized differentiable models demonstrated stronger spatial generalization ability (median KGE 0.64) than a traditional parameter regionalization approach (median KGE 0.46) and even LSTM for ungauged region tests across continents. Nevertheless, relative to LSTM, the differentiable model was hampered by structural deficiencies for cold or polar regions, highly arid regions, and basins with significant human impacts. This study also sets the benchmark for hydrologic estimates around the world and builds a foundation for improving global hydrologic simulations. [ABSTRACT FROM AUTHOR]

Manning's roughness coefficient, n, is used to describe channel roughness, and is a widely sought‐after key parameter for estimating and predicting flood propagation. Due to its control of flow velocity and shear stress, n is critical for modeling timing of floods and pollutants, aquatic ecosystem health, infrastructural safety, and so on. While alternative formulations exist, open‐channel n is typically regarded as temporally constant, determined from lookup tables or calibration, and its spatiotemporal variability was never examined holistically at large scales. Here, we developed and analyzed a continental‐scale n dataset (along with alternative formulations) calculated from observed velocity, slope, and hydraulic radius in 200,000 surveys conducted over 5,000 U.S. sites. These large, diverse observations allowed training of a Random Forest (RF) model capable of predicting n (or alternative parameters) at high accuracy (Nash Sutcliffe model efficiency >0.7) in space and time. We show that predictable time variability explains a large fraction (∼35%) of n variance compared to spatial variability (50%). While exceptions abound, n is generally lower and more stable under higher streamflow conditions. Other factorial influences on n including land cover, sinuosity, and particle sizes largely agree with conventional intuition. Accounting for temporal variability in n could lead to substantially larger (45% at the median site) estimated flow velocities under high‐flow conditions or lower (44%) velocities under low‐flow conditions. Habitual exclusion of n temporal dynamics means flood peaks could arrive days before model‐predicted flood waves, and peak magnitude estimation might also be erroneous. We therefore offer a model of great practical utility. Plain Language Summary: Stream channel roughness is a critical variable for many river‐related applications including modeling of flood inundation extent, pollutant transport, stormwater management, aquatic ecosystem health, infrastructural safety, and so on, and is traditionally assumed as being constant over time. Here we estimate channel roughness using in‐stream measurements from thousands of sites across the United States and show that its temporal dependence can be substantial. Our machine learning model can serve as a valuable and state‐of‐the‐art prediction of roughness, providing great practical value and a holistic view of the spatiotemporal variability of roughness. Moreover, the longstanding exclusion of temporal dynamics means that flood peaks could arrive days before model‐predicted flood waves, and peak magnitude estimation might also be inaccurate. Raising awareness of this issue can advance our understanding of channel flows, improve the accuracy of modeling, and save lives. Key Points: We developed a continental‐scale dataset of Manning's channel roughness values and trained accurate random forest models to predict themPredictable time variability explains a large fraction (∼35%) of variance in Manning's roughness compared to spatial variability (50%)Ignoring temporal dynamics means flood peaks could arrive days before predicted, and peak magnitude estimation might also be erroneous [ABSTRACT FROM AUTHOR]

Recent advances in differentiable modeling, a genre of physics-informed machine learning that trains neural networks (NNs) together with process-based equations, have shown promise in enhancing hydrological models' accuracy, interpretability, and knowledge-discovery potential. Current differentiable models are efficient for NN-based parameter regionalization, but the simple explicit numerical schemes paired with sequential calculations (operator splitting) can incur numerical errors whose impacts on models' representation power and learned parameters are not clear. Implicit schemes, however, cannot rely on automatic differentiation to calculate gradients due to potential issues of gradient vanishing and memory demand. Here we propose a "discretize-then-optimize" adjoint method to enable differentiable implicit numerical schemes for the first time for large-scale hydrological modeling. The adjoint model demonstrates comprehensively improved performance, with Kling–Gupta efficiency coefficients, peak-flow and low-flow metrics, and evapotranspiration that moderately surpass the already-competitive explicit model. Therefore, the previous sequential-calculation approach had a detrimental impact on the model's ability to represent hydrological dynamics. Furthermore, with a structural update that describes capillary rise, the adjoint model can better describe baseflow in arid regions and also produce low flows that outperform even pure machine learning methods such as long short-term memory networks. The adjoint model rectified some parameter distortions but did not alter spatial parameter distributions, demonstrating the robustness of regionalized parameterization. Despite higher computational expenses and modest improvements, the adjoint model's success removes the barrier for complex implicit schemes to enrich differentiable modeling in hydrology. [ABSTRACT FROM AUTHOR]

Predicting the response of hydrologic systems to modified driving forces beyond patterns that have occurred in the past is of high importance for estimating climate change impacts or the effect of management measures. This kind of prediction requires a model, but the impossibility of testing such predictions against observed data makes it difficult to estimate their reliability. Metamorphic testing offers a methodology for assessing models beyond validation with real data. It consists of defining input changes for which the expected responses are assumed to be known, at least qualitatively, and testing model behavior for consistency with these expectations. To increase the gain of information and reduce the subjectivity of this approach, we extend this methodology to a multi-model approach and include a sensitivity analysis of the predictions to training or calibration options. This allows us to quantitatively analyze differences in predictions between different model structures and calibration options in addition to the qualitative test of the expectations. In our case study, we apply this approach to selected conceptual and machine learning hydrological models calibrated for basins from the CAMELS data set. Our results confirm the superiority of the machine learning models over the conceptual hydrologic models regarding the quality of fit during calibration and validation periods. However, we also find that the response of machine learning models to modified inputs can deviate from the expectations and the magnitude, and even the sign of the response can depend on the training data. In addition, even in cases in which all models passed the metamorphic test, there are cases in which the quantitative response is different for different model structures. This demonstrates the importance of this kind of testing beyond and in addition to the usual calibration–validation analysis to identify potential problems and stimulate the development of improved models. [ABSTRACT FROM AUTHOR]

River bathymetry is critical for many aspects of water resources management. We propose and demonstrate a bathymetry inversion method using a deep‐learning‐based surrogate for shallow water equations solvers. The surrogate uses the convolutional autoencoder with a shared‐encoder, separate‐decoder architecture. It encodes the input bathymetry and decodes to separate outputs for flow field variables. Utilizing the differentiability of the surrogate, a gradient‐based optimizer is used to perform bathymetry inversion. Two physically based constraints on the ranges of both bed elevation and slope have to be added as inversion loss regularizations to contract the solution space. Using the "L‐curve" criterion, a heuristic approach is proposed to determine the regularization parameters. Both the surrogate model and the inversion algorithm show good performance. The bathymetry inversion progresses in two distinctive stages, which resembles the sculptural process of initial broad‐brush calving and final fine detailing. The inversion loss due to flow prediction error and the two regularizations play dominant roles in the initial and final stages, respectively. The surrogate architecture (whether with both velocity and water surface elevation or velocity only as outputs) does not have significant impact on inversion result. The methodology proposed in this work, an example of differentiable parameter learning, can be similarly used in the inversion of other important distributed parameters such as roughness coefficient. Key Points: Gradient‐based optimization is used to perform inversion with automatic differentiation enabled by the surrogate based on CNN autoencoderPhysically based regularizations on the ranges of bed elevation and slope are crucial for useable inversion resultsThe methodology works for both simple cases and complex real‐world cases [ABSTRACT FROM AUTHOR]

Recently, rainfall‐runoff simulations in small headwater basins have been improved by methodological advances such as deep neural networks (NNs) and hybrid physics‐NN models—particularly, a genre called differentiable modeling that intermingles NNs with physics to learn relationships between variables. However, hydrologic routing simulations, necessary for simulating floods in stem rivers downstream of large heterogeneous basins, had not yet benefited from these advances and it was unclear if the routing process could be improved via coupled NNs. We present a novel differentiable routing method (δMC‐Juniata‐hydroDL2) that mimics the classical Muskingum‐Cunge routing model over a river network but embeds an NN to infer parameterizations for Manning's roughness (n) and channel geometries from raw reach‐scale attributes like catchment areas and sinuosity. The NN was trained solely on downstream hydrographs. Synthetic experiments show that while the channel geometry parameter was unidentifiable, n can be identified with moderate precision. With real‐world data, the trained differentiable routing model produced more accurate long‐term routing results for both the training gage and untrained inner gages for larger subbasins (>2,000 km2) than either a machine learning model assuming homogeneity, or simply using the sum of runoff from subbasins. The n parameterization trained on short periods gave high performance in other periods, despite significant errors in runoff inputs. The learned n pattern was consistent with literature expectations, demonstrating the framework's potential for knowledge discovery, but the absolute values can vary depending on training periods. The trained n parameterization can be coupled with traditional models to improve national‐scale hydrologic flood simulations. Key Points: A novel differentiable routing model can learn effective river routing parameterization, recovering channel roughness in synthetic runsWith short periods of real training data, we can improve streamflow in large rivers compared to models not considering routingFor basins >2,000 km2, our framework outperformed deep learning models that assume homogeneity, despite bias in the runoff forcings [ABSTRACT FROM AUTHOR]

Accurate prediction of snow water equivalent (SWE) can be valuable for water resource managers. Recently, deep learning methods such as long short-term memory (LSTM) have exhibited high accuracy in simulating hydrologic variables and can integrate lagged observations to improve prediction, but their benefits were not clear for SWE simulations. Here we tested an LSTM network with data integration (DI) for SWE in the western United States to integrate 30-day-lagged or 7-day-lagged observations of either SWE or satellite-observed snow cover fraction (SCF) to improve future predictions. SCF proved beneficial only for shallow-snow sites during snowmelt, while lagged SWE integration significantly improved prediction accuracy for both shallow- and deep-snow sites. The median Nash–Sutcliffe model efficiency coefficient (NSE) in temporal testing improved from 0.92 to 0.97 with 30-day-lagged SWE integration, and root-mean-square error (RMSE) and the difference between estimated and observed peak SWE values dmax were reduced by 41% and 57%, respectively. DI effectively mitigated accumulated model and forcing errors that would otherwise be persistent. Moreover, by applying DI to different observations (30-day-lagged, 7-day-lagged), we revealed the spatial distribution of errors with different persistent lengths. For example, integrating 30-day-lagged SWE was ineffective for ephemeral snow sites in the southwestern United States, but significantly reduced monthly-scale biases for regions with stable seasonal snowpack such as high-elevation sites in California. These biases are likely attributable to large interannual variability in snowfall or site-specific snow redistribution patterns that can accumulate to impactful levels over time for nonephemeral sites. These results set up benchmark levels and provide guidance for future model improvement strategies. [ABSTRACT FROM AUTHOR]

Although deep learning models for stream temperature (Ts) have recently shown exceptional accuracy, they have limited interpretability and cannot output untrained variables. With hybrid differentiable models, neural networks (NNs) can be connected to physically based equations (called structural priors) to output intermediate variables such as water source fractions (specifying what portion of water is groundwater, subsurface, and surface flow). However, it is unclear if such outputs are physically meaningful when only limited physics is imposed, and if structural priors have enough impacts to be identifiable from data. Here, we tested four alternative structural priors describing basin‐scale water temperature memory and instream heat processes in a differentiable stream temperature model where NNs freely estimate the water source fractions. We evaluated models' abilities to predict Ts and baseflow ratio. The four priors exhibited noticeably different behaviors in these two metrics and their tradeoffs, with some dominating others. Therefore, the better structural priors can be identified. Moreover, testing different priors yielded valuable insights: having a separate shallow subsurface flow component better matches observations, and a recency‐weighted averaging of past air temperature for calculating source water temperature resulted in better Ts and baseflow prediction than traditionally employed simple averaging. However, we also highlight the limitations when insufficient physical constraints are implemented: the internal variables (water source fractions) may not be adequately constrained by a single target variable (stream temperature) alone. To ensure the physical significance of the internal fluxes, one can either employ multivariate data for model selection, or include more physical processes in the priors. Plain Language Summary: A new framework called differentiable modeling combines the benefits from neural networks (NNs) and process‐based models. This framework can learn from big data while the process‐based model components (called prior knowledge, or priors) are intended to output intermediate physical variables. However, do such priors matter, can we tell if one set of priors is better than another, and do the intermediate outputs represent the intended physical concepts? We explore these questions with a differentiable stream temperature model where the NN replaces the hydrologic component and estimates parameters pertaining to the stream temperature module. The strong optimizing capability of NNs allows us to avoid some complexities and attribute the differences in model outcomes to the assumed priors. Testing different priors thus yielded many important lessons, for example, the need for having a separate shallow groundwater "bucket," the benefit of placing more importance on recent air temperature when estimating groundwater temperature, and the importance of describing in‐stream temperature. The results show lots of untapped potential with differentiable modeling and the data we have available. Key Points: We can identify better structural priors (process equations) using differentiable models, circumventing intertwined parameter issuesConsidering a separate bucket for shallow subsurface water improves both stream temperature and baseflow simulationThe models selected by the multivariate evaluation produce physically meaningful estimates of water source fractions [ABSTRACT FROM AUTHOR]

Transportation corridors constructed using through- and side-cuts are susceptible to hazardous slope failures, potentially causing infrastructure damage, operational suspensions and loss of life. To monitor the stability of known geohazards at the local scale, geotechnical investigation of each slope is typically performed to calculate a factor of safety. In many corridors, however, this method is labour-intensive due to the quantity of geohazards and statistical methods are instead used to identify hazardous sections. This paper introduces a new slope failure hazard assessment technique, utilising susceptibility mapping of geospatial information and computer vision-based analysis of right-of-way videos recorded by railroad track inspection vehicles, applied to a section of railroad track near Harrisburg, Pennsylvania. Combining these results, an enhanced relative hazard assessment algorithm was formulated. Using the developed framework, geohazards of primary concern were determined which should be prioritised for future geotechnical investigation and remediation efforts. [ABSTRACT FROM AUTHOR]

Physics-based models (PBMs), such as shallow water equations (SWEs) solvers, have been widely used in flood simulation and river hydraulics analysis. However, they are usually computationally expensive and unsuitable for parameter optimizations that need many runs. An alternative is the machine learning (ML) method, which can be used to construct computationally efficient surrogates for PBMs that can approximate their input-output dynamics. Among many ML techniques, convolutional neural network (CNN) is a prevalent method for image-to-image regressions on structured or regular meshes (e.g., mapping from the boundary conditions to flow solutions of SWEs). However, CNN-based methods have significant limitations because of their raster-image nature. Such methods cannot precisely capture the boundary geometry of obstacles and near-field flow features, which are of paramount importance to fluid dynamics. We introduced an efficient, accurate, and flexible neural network (NN) surrogate model [which is based on deep learning and can make point-to-point (p2p) predictions on unstructured meshes] called NN-p2p. The new method was evaluated and compared against CNN-based methods. NN-p2p improves the accuracy of the near-field flow prediction with a mean relative error of 0.56% for the velocity magnitude around piers with unseen length/width ratios. It also respects conservation laws more strictly than the CNN-based models and performs reasonably well for spatial extrapolation. The surrogate reduces computing time by almost 3-orders of magnitude in comparison with its corresponding PBM. Moreover, as a demonstration of the NN-p2p model's practical applicability, we calculated drag coefficient using NN-p2p for piers of varying length-to-width ratios and obtained a novel linear relationship between the drag coefficient and the logarithmic transformation of the pier's geometry. [ABSTRACT FROM AUTHOR]

Slope stability prediction is an important task in geotechnical engineering which can be achieved through physics-based or data-driven approaches. Physics-based approaches rely on geotechnical knowledge from soil mechanics, such as limit equilibrium analysis and shear strength theories, to evaluate the stability condition of slopes, and they are often limited to slope-specific analysis. Data-driven approaches predict slope stability conditions based on learned relationships between influencing factors and slope stability conditions from past observations of slope failures (i.e., case histories); they rely on big data which are difficult to obtain. This study examines three easy-to-implement and effective methods to integrate geotechnical engineering domain knowledge into data-driven models for slope stability prediction: hybrid knowledge-data model, knowledge-based model initiation, and knowledge-guided loss function. These models were benchmarked against pure data-driven models and domain knowledge–based models, including a physics-based solution chart and a physics-based empirical model. A compilation of slope stability case histories from the literature was used as the benchmark database, and five-fold cross-validation was employed to evaluate model performance. The model validation results demonstrated that machine learning models outperformed domain knowledge–based models in terms of several evaluation metrics. The three proposed methods were found to outperform both domain knowledge–based models and pure data-driven models. Additionally, the hybrid knowledge-data models and knowledge-guided loss function were found to reduce discrepancies in the predicted slope stability conditions compared with reported factor-of-safety values, leading to a better alignment with the underlying physics related to slope stability. This study provides an initial assessment of the value of coupling domain knowledge and data-driven methods in geotechnical engineering applications using slope stability prediction as an example. [ABSTRACT FROM AUTHOR]

Photosynthesis plays an important role in carbon, nitrogen, and water cycles. Ecosystem models for photosynthesis are characterized by many parameters that are obtained from limited in situ measurements and applied to the same plant types. Previous site-by-site calibration approaches could not leverage big data and faced issues like overfitting or parameter non-uniqueness. Here we developed an end-to-end programmatically differentiable (meaning gradients of outputs to variables used in the model can be obtained efficiently and accurately) version of the photosynthesis process representation within the Functionally Assembled Terrestrial Ecosystem Simulator (FATES) model. As a genre of physics-informed machine learning (ML), differentiable models couple physics-based formulations to neural networks (NNs) that learn parameterizations (and potentially processes) from observations, here photosynthesis rates. We first demonstrated that the framework was able to correctly recover multiple assumed parameter values concurrently using synthetic training data. Then, using a real-world dataset consisting of many different plant functional types (PFTs), we learned parameters that performed substantially better and greatly reduced biases compared to literature values. Further, the framework allowed us to gain insights at a large scale. Our results showed that the carboxylation rate at 25 ∘ C (Vc,max25) was more impactful than a factor representing water limitation, although tuning both was helpful in addressing biases with the default values. This framework could potentially enable substantial improvement in our capability to learn parameters and reduce biases for ecosystem modeling at large scales. [ABSTRACT FROM AUTHOR]

Accurate hydrological predictions are required to prepare for the impacts of climate change, especially in India, which experiences frequent floods and droughts. However, the complex hydrological processes of its distinct watersheds and limited data make it challenging to deliver highly‐performant hydrologic predictions using conventional models. Moreover, it remains uncertain where the limits of predictability are and whether recently‐popular deep learning approaches can offer significant improvements. Here, we tested the first instance of the hydrologic model based on long short‐term memory (LSTM) for 55 Indian watersheds, using a new dataset comprising forcing, attributes, and discharge data. Our results show that the LSTM model provides much‐improved performance compared to conventional models in India, providing a median Nash‐Sutcliffe efficiency (NSE) of 0.56. The LSTM model trained on all the watersheds is more favourable to those trained on individual or homogeneous watersheds, as it benefits from a broader range of hydrological processes and patterns in the input data. However, the LSTM model performs poorly for non‐perennial, large, and semi‐arid climate zone watersheds due to its inability to simulate the complex hydrological processes specific to these environments. Integrating lagged observations with the LSTM model (referred to as DI‐LSTM) improved the predictions in such watersheds and enhanced the median NSE to 0.76 by capturing the temporal dependencies and historical patterns that influence hydrological processes. Overall, the contrast of model performance across watersheds suggests major limitations could be associated with the quality of forcing data, and the slow flow or groundwater processes are highly important in the Indian subcontinent. Notably, both LSTM and DI‐LSTM models performed reasonably well for predictions in ungauged watersheds. The findings of this study demonstrate that data‐sparse countries, too, can benefit from big‐data deep learning and point out further avenues toward model improvements. [ABSTRACT FROM AUTHOR]

As a genre of physics-informed machine learning, differentiable process-based hydrologic models (abbreviated as δ or delta models) with regionalized deep-network-based parameterization pipelines were recently shown to provide daily streamflow prediction performance closely approaching that of state-of-the-art long short-term memory (LSTM) deep networks. Meanwhile, δ models provide a full suite of diagnostic physical variables and guaranteed mass conservation. Here, we ran experiments to test (1) their ability to extrapolate to regions far from streamflow gauges and (2) their ability to make credible predictions of long-term (decadal-scale) change trends. We evaluated the models based on daily hydrograph metrics (Nash–Sutcliffe model efficiency coefficient, etc.) and predicted decadal streamflow trends. For prediction in ungauged basins (PUB; randomly sampled ungauged basins representing spatial interpolation), δ models either approached or surpassed the performance of LSTM in daily hydrograph metrics, depending on the meteorological forcing data used. They presented a comparable trend performance to LSTM for annual mean flow and high flow but worse trends for low flow. For prediction in ungauged regions (PUR; regional holdout test representing spatial extrapolation in a highly data-sparse scenario), δ models surpassed LSTM in daily hydrograph metrics, and their advantages in mean and high flow trends became prominent. In addition, an untrained variable, evapotranspiration, retained good seasonality even for extrapolated cases. The δ models' deep-network-based parameterization pipeline produced parameter fields that maintain remarkably stable spatial patterns even in highly data-scarce scenarios, which explains their robustness. Combined with their interpretability and ability to assimilate multi-source observations, the δ models are strong candidates for regional and global-scale hydrologic simulations and climate change impact assessment. [ABSTRACT FROM AUTHOR]

Hybrid hydroclimatic forecasting systems employ data-driven (statistical or machine learning) methods to harness and integrate a broad variety of predictions from dynamical, physics-based models – such as numerical weather prediction, climate, land, hydrology, and Earth system models – into a final prediction product. They are recognized as a promising way of enhancing the prediction skill of meteorological and hydroclimatic variables and events, including rainfall, temperature, streamflow, floods, droughts, tropical cyclones, or atmospheric rivers. Hybrid forecasting methods are now receiving growing attention due to advances in weather and climate prediction systems at subseasonal to decadal scales, a better appreciation of the strengths of AI, and expanding access to computational resources and methods. Such systems are attractive because they may avoid the need to run a computationally expensive offline land model, can minimize the effect of biases that exist within dynamical outputs, benefit from the strengths of machine learning, and can learn from large datasets, while combining different sources of predictability with varying time horizons. Here we review recent developments in hybrid hydroclimatic forecasting and outline key challenges and opportunities for further research. These include obtaining physically explainable results, assimilating human influences from novel data sources, integrating new ensemble techniques to improve predictive skill, creating seamless prediction schemes that merge short to long lead times, incorporating initial land surface and ocean/ice conditions, acknowledging spatial variability in landscape and atmospheric forcing, and increasing the operational uptake of hybrid prediction schemes. [ABSTRACT FROM AUTHOR]

Climate change threatens our ability to grow food for an ever-increasing population. There is a need for high-quality soil moisture predictions in under-monitored regions like Africa. However, it is unclear if soil moisture processes are globally similar enough to allow our models trained on available in situ data to maintain accuracy in unmonitored regions. We present a multitask long short-term memory (LSTM) model that learns simultaneously from global satellite-based data and in situ soil moisture data. This model is evaluated in both random spatial holdout mode and continental holdout mode (trained on some continents, tested on a different one). The model compared favorably to current land surface models, satellite products, and a candidate machine learning model, reaching a global median correlation of 0.792 for the random spatial holdout test. It behaved surprisingly well in Africa and Australia, showing high correlation even when we excluded their sites from the training set, but it performed relatively poorly in Alaska where rapid changes are occurring. In all but one continent (Asia), the multitask model in the worst-case scenario test performed better than the soil moisture active passive (SMAP) 9 km product. Factorial analysis has shown that the LSTM model's accuracy varies with terrain aspect, resulting in lower performance for dry and south-facing slopes or wet and north-facing slopes. This knowledge helps us apply the model while understanding its limitations. This model is being integrated into an operational agricultural assistance application which currently provides information to 13 million African farmers. [ABSTRACT FROM AUTHOR]

Predictions of hydrologic variables across the entire water cycle have significant value for water resources management as well as downstream applications such as ecosystem and water quality modeling. Recently, purely data‐driven deep learning models like long short‐term memory (LSTM) showed seemingly insurmountable performance in modeling rainfall runoff and other geoscientific variables, yet they cannot predict untrained physical variables and remain challenging to interpret. Here, we show that differentiable, learnable, process‐based models (called δ models here) can approach the performance level of LSTM for the intensively observed variable (streamflow) with regionalized parameterization. We use a simple hydrologic model HBV as the backbone and use embedded neural networks, which can only be trained in a differentiable programming framework, to parameterize, enhance, or replace the process‐based model's modules. Without using an ensemble or post‐processor, δ models can obtain a median Nash‐Sutcliffe efficiency of 0.732 for 671 basins across the USA for the Daymet forcing data set, compared to 0.748 from a state‐of‐the‐art LSTM model with the same setup. For another forcing data set, the difference is even smaller: 0.715 versus 0.722. Meanwhile, the resulting learnable process‐based models can output a full set of untrained variables, for example, soil and groundwater storage, snowpack, evapotranspiration, and baseflow, and can later be constrained by their observations. Both simulated evapotranspiration and fraction of discharge from baseflow agreed decently with alternative estimates. The general framework can work with models with various process complexity and opens up the path for learning physics from big data. Plain Language Summary: Recently, deep neural networks like long short‐term memory (LSTM) have received a lot of attention for producing high‐accuracy simulations in hydrology, but they do not respect physical laws and remain difficult to understand. However, what if you can have a model with similar accuracy, but with clarity about physical processes? What if at the same time the model respects physical laws like mass conservation and produces interpretable outputs (like soil moisture, groundwater storage, evapotranspiration and baseflow), with which you can tell a whole story to stakeholders? What if the same framework allows you to ask precise questions about different parts of hydrology and re‐examine your understanding of some parts of the physical system or check if your past equations are correct? This paper delivers a system that achieves these grand goals and opens many avenues for further exploration. Key Points: Differentiable (δ) hydrologic models show that process clarity and a performance approaching deep learning (DL) can be both attainedUnlike DL, δ models can output untrained physical variables, which agreed well with alternative estimatesDynamic parameterization has a moderate impact but is needed to narrow the gap to DL, suggesting that current model states are inadequate [ABSTRACT FROM AUTHOR]

A warming climate will intensify the water cycle, resulting in an exacerbation of water resources crises and flooding risks in the Lancang–Mekong River Basin (LMRB). The mitigation of these risks requires accurate streamflow and flood simulations. Process-based and data-driven hydrological models are the two major approaches for streamflow simulations, while a hybrid of these two methods promises advantageous prediction accuracy. In this study, we developed a hybrid physics-data (HPD) methodology for streamflow and flood prediction under the physics-guided neural network modeling framework. The HPD methodology leveraged simulation information from a process-based model (i.e., VIC-CaMa-Flood) along with the meteorological forcing information (precipitation, maximum temperature, minimum temperature, and wind speed) to simulate the daily streamflow series and flood events, using a long short-term memory (LSTM) neural network. This HPD methodology outperformed the pure process-based VIC-CaMa-Flood model or the pure observational data driven LSTM model by a large margin, suggesting the usefulness of introducing physical regularization in data-driven modeling, and the necessity of observation-informed bias correction for process-based models. We further developed a gradient boosting tree method to measure the information contribution from the process-based model simulation and the meteorological forcing data in our HPD methodology. The results show that the process-based model simulation contributes about 30% to the HPD outcome, outweighing the information contribution from each of the meteorological forcing variables (<20%). Our HPD methodology inherited the physical mechanisms of the process-based model, and the high predictability capability of the LSTM model, offering a novel way for making use of incomplete physical understanding, and insufficient data, to enhance streamflow and flood predictions. [ABSTRACT FROM AUTHOR]

Deep learning (DL) models trained on hydrologic observations can perform extraordinarily well, but they can inherit deficiencies of the training data, such as limited coverage of in situ data or low resolution/accuracy of satellite data. Here we propose a novel multiscale DL scheme learning simultaneously from satellite and in situ data to predict 9 km daily soil moisture (5 cm depth). Based on spatial cross‐validation over sites in the conterminous United States, the multiscale scheme obtained a median correlation of 0.901 and root‐mean‐square error of 0.034 m3/m3. It outperformed the Soil Moisture Active Passive satellite mission's 9 km product, DL models trained on in situ data alone, and land surface models. Our 9 km product showed better accuracy than previous 1 km satellite downscaling products, highlighting limited impacts of improving resolution. Not only is our product useful for planning against floods, droughts, and pests, our scheme is generically applicable to geoscientific domains with data on multiple scales, breaking the confines of individual data sets. Plain Language Summary: High‐resolution soil moisture data can be of great value for many practical applications, for example, flood forecasting, drought monitoring, crop management, weather forecasting, and better understanding the world's ecosystems. Recently, deep learning (DL) models directly learning patterns from observations have shown strong performance in modeling soil moisture and other environmental factors. Used directly, however, they inherit certain limitations of their training data such as low resolution and low accuracy; in other words, these models are students that cannot exceed their teacher (the data source). For soil moisture, just as for many other environmental variables of interest, observations are available on multiple scales, for example, probes in the ground and satellite‐based data. Here we show that learning from multiple data sources on different scales at the same time can allow DL models to overcome limitations of every single source: the student model can then outperform each teacher. Our multiscale model reported the best metrics for daily soil moisture prediction compared to alternatives and is no longer limited by poor satellite sensing capability in certain regions. This multiscale scheme is broadly applicable to many areas of environmental study. Key Points: We propose a novel multiscale soil moisture model (5 cm depth) learning from both satellite and in situ data, resulting in high accuracyIn situ data provides more information than satellite data, but if used together, they can overcome the limitations of each data sourceThe impacts of input resolution waned below the 9 km grid, and the results established the upper error bound (RMSE ∼0.034) due to resolution [ABSTRACT FROM AUTHOR]

When fitting statistical models to variables in geoscientific disciplines such as hydrology, it is a customary practice to stratify a large domain into multiple regions (or regimes) and study each region separately. Traditional wisdom suggests that models built for each region separately will have higher performance because of homogeneity within each region. However, each stratified model has access to fewer and less diverse data points. Here, through two hydrologic examples (soil moisture and streamflow), we show that conventional wisdom may no longer hold in the era of big data and deep learning (DL). We systematically examined an effect we call data synergy, where the results of the DL models improved when data were pooled together from characteristically different regions. The performance of the DL models benefited from modest diversity in the training data compared to a homogeneous training set, even with similar data quantity. Moreover, allowing heterogeneous training data makes eligible much larger training datasets, which is an inherent advantage of DL. A large, diverse data set is advantageous in terms of representing extreme events and future scenarios, which has strong implications for climate change impact assessment. The results here suggest the research community should place greater emphasis on data sharing. Plain Language Summary: Traditionally with statistical methods used in hydrology, we split the domain into relatively homogeneous regimes, for each of which we can create a simple model, that is, a local model. However, in the era of big data machine learning, we show that this is often the opposite of what should be done. With deep learning models, we should compile a large and heterogeneous data set and compare the local model to a model trained with all the data (global model). Including heterogeneous training samples may improve the results compared to the local model. We call this the data synergy effect, and it results from two main factors. First, deep learning models are complex enough to accommodate different training instances, inherently permitting larger training datasets with more extreme events and changing trends. Second, with a heterogeneous training data set, deep learning models may be able to learn both the underlying similarities and factors contributing to differences between regions. Key Points: We introduced data synergy, where deep learning performance in a local region improves when including samples from other regionsData synergy is apparent with modestly diverse training data, partly because a larger and more diverse data set contains more extreme eventsThis work highlighted the value of samples outside a region of interest, emphasizing the need for community data sharing [ABSTRACT FROM AUTHOR]

Downward longwave radiation (DLR) at the surface is a key variable of interest in fields, such as hydrology and climate research. However, existing DLR estimation methods and DLR products are still problematic in terms of both accuracy and spatiotemporal resolution. In this article, we propose a deep convolutional neural network (DCNN)-based method to estimate hourly DLR at 5-km spatial resolution from top of atmosphere (TOA) brightness temperature (BT) of the Himawari-8/Advanced Himawari Imager (AHI) thermal channels, combined with near-surface air temperature and dew point temperature of ERA5 and elevation data. Validation results show that the DCNN-based method outperforms popular random forest and multilayer perceptron-based methods and that our proposed scheme integrating multisource data outperforms that only using remote sensing TOA observations or surface meteorological data. Compared with state-of-the-art CERES-SYN and ERA5-land DLR products, the estimated DLR by our proposed DCNN-based method with physical multisource inputs has higher spatiotemporal resolution and accuracy, with correlation coefficient (CC) of 0.95, root-mean-square error (RMSE) of 17.2 W/m2, and mean bias error (MBE) of −0.8 W/m2 in the testing period on the Tibetan Plateau. [ABSTRACT FROM AUTHOR]

Basin‐centric long short‐term memory (LSTM) network models have recently been shown to be an exceptionally powerful tool for stream temperature (Ts) temporal prediction (training in one period and predicting in another period at the same sites). However, spatial extrapolation is a well‐known challenge to modelling Ts and it is uncertain how an LSTM‐based daily Ts model will perform in unmonitored or dammed basins. Here we compiled a new benchmark dataset consisting of >400 basins across the contiguous United States in different data availability groups (DAG, meaning the daily sampling frequency) with and without major dams, and studied how to assemble suitable training datasets for predictions in basins with or without temperature monitoring. For prediction in unmonitored basins (PUB), LSTM produced a root‐mean‐square error (RMSE) of 1.129°C and an R2 of 0.983. While these metrics declined from LSTM's temporal prediction performance, they far surpassed traditional models' PUB values, and were competitive with traditional models' temporal prediction on calibrated sites. Even for unmonitored basins with major reservoirs, we obtained a median RMSE of 1.202°C and an R2 of 0.984. For temporal prediction, the most suitable training set was the matching DAG that the basin could be grouped into (for example, the 60% DAG was most suitable for a basin with 61% data availability). However, for PUB, a training dataset including all basins with data was consistently preferred. An input‐selection ensemble moderately mitigated attribute overfitting. Our results indicate there are influential latent processes not sufficiently described by the inputs (e.g., geology, wetland covers), but temporal fluctuations can still be predicted well, and LSTM appears to be a highly accurate Ts modelling tool even for spatial extrapolation. [ABSTRACT FROM AUTHOR]