Search

AmeriFlux is a network of research sites that measure carbon, water, and energy fluxes between ecosystems and the atmosphere using the eddy covariance technique to study a variety of Earth science questions. AmeriFlux's diversity of ecosystems, instruments, and data-processing routines create challenges for data standardization, quality assurance, and sharing across the network. To address these challenges, the AmeriFlux Management Project (AMP) designed and implemented the BASE data-processing pipeline. The pipeline begins with data uploaded by the site teams, followed by the AMP team's quality assurance and quality control (QA/QC), ingestion of site metadata, and publication of the BASE data product. The semi-automated pipeline enables us to keep pace with the rapid growth of the network. As of 2022, the AmeriFlux BASE data product contains 3,130 site years of data from 444 sites, with standardized units and variable names of more than 60 common variables, representing the largest long-term data repository for flux-met data in the world. The standardized, quality-ensured data product facilitates multisite comparisons, model evaluations, and data syntheses.

The global decline of water quality in rivers and streams has resulted in a pressing need to design new watershed management strategies. Water quality can be affected by multiple stressors including population growth, land use change, global warming, and extreme events, with repercussions on human and ecosystem health. A scientific understanding of factors affecting riverine water quality and predictions at local to regional scales, and at sub-daily to decadal timescales are needed for optimal management of watersheds and river basins. Here, we discuss how machine learning (ML) can enable development of more accurate, computationally tractable, and scalable models for analysis and predictions of river water quality. We review relevant state-of-the art applications of ML for water quality models and discuss opportunities to improve the use of ML with emerging computational and mathematical methods for model selection, hyperparameter optimization, incorporating process knowledge into ML models, improving explainablity, uncertainty quantification, and model-data integration. We then present considerations for using ML to address water quality problems given their scale and complexity, available data and computational resources, and stakeholder needs. When combined with decades of process understanding, interdisciplinary advances in knowledge-guided ML, information theory, data integration, and analytics can help address fundamental science questions and enable decision-relevant predictions of riverine water quality.

Diverse observational and simulation datasets are needed to understand and predict complex ecosystem behavior over seasonal to decadal and century time-scales. Integration of these datasets poses a major barrier towards advancing environmental science, particularly due to differences in the structure and formats of data provided by various sources. Here, we describe BASIN-3D (Broker for Assimilation, Synthesis and Integration of eNvironmental Diverse, Distributed Datasets), a data integration framework designed to dynamically retrieve and transform heterogeneous data from different sources into a common format to provide an integrated view. BASIN-3D enables users to adopt a standardized approach for data retrieval and avoid customizations for the data type or source. We demonstrate the value of BASIN-3D with two use cases that require integration of data from regional to watershed spatial scales. The first application uses the BASIN-3D Python library to integrate time-series hydrological and meteorological data to provide standardized inputs to analytical and machine learning codes in order to predict the impacts of hydrological disturbances on large river corridors of the United States. The second application uses the BASIN-3D Django framework to integrate diverse time-series data in a mountainous watershed in East River, Colorado, United States to enable scientific researchers to explore and download data through an interactive web portal. Thus, BASIN-3D can be used to support data integration for both web-based tools, as well as data analytics using Python scripting and extensions like Jupyter notebooks. The framework is expected to be transferable to and useful for many other field and modeling studies.

Research can be more transparent and collaborative by using Findable, Accessible, Interoperable, and Reusable (FAIR) principles to publish Earth and environmental science data. Reporting formats-instructions, templates, and tools for consistently formatting data within a discipline-can help make data more accessible and reusable. However, the immense diversity of data types across Earth science disciplines makes development and adoption challenging. Here, we describe 11 community reporting formats for a diverse set of Earth science (meta)data including cross-domain metadata (dataset metadata, location metadata, sample metadata), file-formatting guidelines (file-level metadata, CSV files, terrestrial model data archiving), and domain-specific reporting formats for some biological, geochemical, and hydrological data (amplicon abundance tables, leaf-level gas exchange, soil respiration, water and sediment chemistry, sensor-based hydrologic measurements). More broadly, we provide guidelines that communities can use to create new (meta)data formats that integrate with their scientific workflows. Such reporting formats have the potential to accelerate scientific discovery and predictions by making it easier for data contributors to provide (meta)data that are more interoperable and reusable.

Stream temperature (Ts) is an important water quality parameter that affects ecosystem health and human water use for beneficial purposes. Accurate Ts predictions at different spatial and temporal scales can inform water management decisions that account for the effects of changing climate and extreme events. In particular, widespread predictions of Ts in unmonitored stream reaches can enable decision makers to be responsive to changes caused by unforeseen disturbances. In this study, we demonstrate the use of classical machine learning (ML) models, support vector regression and gradient boosted trees (XGBoost), for monthly Ts predictions in 78 pristine and human-impacted catchments of the Mid-Atlantic and Pacific Northwest hydrologic regions spanning different geologies, climate, and land use. The ML models were trained using long-term monitoring data from 1980–2020 for three scenarios: (1) temporal predictions at a single site, (2) temporal predictions for multiple sites within a region, and (3) spatiotemporal predictions in unmonitored basins (PUB). In the first two scenarios, the ML models predicted Ts with median root mean squared errors (RMSE) of 0.69–0.84 °C and 0.92–1.02 °C across different model types for the temporal predictions at single and multiple sites respectively. For the PUB scenario, we used a bootstrap aggregation approach using models trained with different subsets of data, for which an ensemble XGBoost implementation outperformed all other modeling configurations (median RMSE 0.62 °C).The ML models improved median monthly Ts estimates compared to baseline statistical multi-linear regression models by 15–48% depending on the site and scenario. Air temperature was found to be the primary driver of monthly Ts for all sites, with secondary influence of month of the year (seasonality) and solar radiation, while discharge was a significant predictor at only 10 sites. The predictive performance of the ML models was robust to configuration changes in model setup and inputs, but was influenced by the distance to the nearest dam with RMSE

The National Microbiome Data Collaborative (NMDC) Data Portal (https://data.microbiomedata.org) supports microbiome multi-omics data exploration and access through an integrated, distributed data framework aligned with the FAIR (Findable, Accessible, Interoperable and Reusable) data principles (1). The NMDC Data Portal currently hosts 10.2 terabytes of multi-omics microbiome data, spanning five data types (metagenomes, metatranscriptomes, metaproteomes, metabolomes, and natural organic matter characterizations), generated at two Department of Energy User Facilities, the Joint Genome Institute (JGI) at Lawrence Berkeley National Laboratory (LBNL) and the Environmental Molecular Systems Laboratory (EMSL) at Pacific Northwest National Laboratory (PNNL). A flexible data schema (https://github.com/microbiomedata/nmdc-schema) leveraging community-driven standards underpins how data is managed and integrated. Annotated multi-omic data products are produced by the NMDC workflows and linked through common biosamples to enable search capabilities based on environmental context, instrumentation, and functional attributes. As a pilot system, the NMDC Data Portal offers download capabilities and several search components, including interactive geographic visualization of samples; environmental classification distribution visualized through an interactive Sankey diagram; time-series slider to select longitudinal samples of interest; and an upset plot displaying the number of multi-omics data generated from the same biosample within a study.

The U.S. Department of Energy's (DOE) Colorado East River Community Observatory (ER) in the Upper Colorado River Basin was established in 2015 as a representative mountainous, snow-dominated watershed to study hydrobiogeochemical responses to hydrological perturbations in headwater systems. The ER is characterized by steep elevation, geologic, hydrologic and vegetation gradients along floodplain, montane, subalpine, and alpine life zones, which makes it an ideal location for researchers to understand how different mountain subsystems contribute to overall watershed behaviour. The ER has both long-term and spatially-extensive observations and experimental campaigns carried out by the Watershed Function Scientific Focus Area (SFA), led by Lawrence Berkeley National Laboratory, and researchers from over 30 organizations who conduct cross-disciplinary process-based investigations and modelling of watershed behaviour. The heterogeneous data generated at the ER include hydrological, genomic, biogeochemical, climate, vegetation, geological, and remote sensing data, which combined with model inputs and outputs comprise a collection of datasets and value-added products within a mountainous watershed that span multiple spatiotemporal scales, compartments, and life zones. Within 5 years of collection, these datasets have revealed insights into numerous aspects of watershed function such as factors influencing snow accumulation and melt timing, water balance partitioning, and impacts of floodplain biogeochemistry and hillslope ecohydrology on riverine geochemical exports. Data generated by the SFA are managed and curated through its Data Management Framework. The SFA has an open data policy, and over 70 ER datasets are publicly available through relevant data repositories. A public interactive map of data collection sites run by the SFA is available to inform the broader community about SFA field activities. Here, we describe the ER and the SFA measurement network, present the public data collection generated by the SFA and partner institutions, and highlight the value of collecting multidisciplinary multiscale measurements in representative catchment observatories.

Field observations of the soil-to-atmosphere CO2 flux—soil respiration, RS—are a prime example of ‘long tail’ data that historically have had neither centralized databases nor an agreed-upon reporting format. This has hindered scientific transparency, analytical reproducibility, and syntheses with respect to this globally-important component of the carbon cycle. Here we propose a new data and metadata reporting format for RS data, based on engagement with a wide range of researchers in the earth and ecological sciences as well as expert advisory panels. Our goal was a reporting format that would be relevant and useful for synthesis activities, optimizing data discoverability and usability while not placing an undue burden on data contributors. We describe previous RS data collection efforts, lessons learned from related databases and data-oriented networks (e.g., FLUXNET) in earth and ecological sciences, and the process of community consultation. The proposed reporting format focuses on chamber-level data and metadata, specifying measurement conditions and, for a given measurement period defined by beginning and ending timestamps, a mean RS flux (or CO2 concentration) and associated ancillary measurements. With input from the research community, we have also developed research data and metadata templates to support data collection adhering to the reporting format. Fundamentally, this format aims to enable findable, accessible, interoperable, and reusable data, while providing ‘future-proofing’ capabilities to support reanalyses using as yet unknown algorithms or approaches. This proposed RS reporting format is openly available online and is intended to be a dynamic document, subject to further community feedback and/or change.

Recent emphasis and requirements for open data publication have led to significant increases in data availability in the Earth sciences, which is critical to long-tail data integration. Currently, data are often published in a repository with an identifier and citation, similar to those for papers. Subsequent publications that use the data are expected to provide a citation in the reference section of the paper. However, the format of the data citation is still evolving, particularly with regards to citing dynamic data, subsets, and collections of data. Considering the motivations of both data producers and consumers, the most pressing need is to create user-friendly solutions that provide credit for data producers and enable accurate citation of data, particularly integrated data. Providing easy-to-use data citations is a critical foundation that is required to address the socio-technical challenges around data integration. Studies that integrate data from dozens or hundreds of datasets must often include data citations in supplementary material due to page limits. However, citations in the supplementary material are not indexed, making it difficult to track citations and thus giving credit to the data producer. In this paper, we discuss our experiences and the challenges we have encountered with current citation guidance. We also review the relative merits of the currently available mechanisms designed to enable compact citation of collections of data, such as data collections, data papers, and dynamic data citations. We consider these options for three data producer scenarios: a domain-specific data collection, a data repository, and a large-scale, multidisciplinary project. We posit that a new mechanism is also needed to enable citation of multiple datasets and credit to data producers.

Globally, soils store two to three times as much carbon as currently resides in the atmosphere, and it is critical to understand how soil greenhouse gas (GHG) emissions and uptake will respond to ongoing climate change. In particular, the soil-to-atmosphere CO2 flux, commonly though imprecisely termed soil respiration (RS ), is one of the largest carbon fluxes in the Earth system. An increasing number of high-frequency RS measurements (typically, from an automated system with hourly sampling) have been made over the last two decades; an increasing number of methane measurements are being made with such systems as well. Such high frequency data are an invaluable resource for understanding GHG fluxes, but lack a central database or repository. Here we describe the lightweight, open-source COSORE (COntinuous SOil REspiration) database and software, that focuses on automated, continuous and long-term GHG flux datasets, and is intended to serve as a community resource for earth sciences, climate change syntheses and model evaluation. Contributed datasets are mapped to a single, consistent standard, with metadata on contributors, geographic location, measurement conditions and ancillary data. The design emphasizes the importance of reproducibility, scientific transparency and open access to data. While being oriented towards continuously measured RS , the database design accommodates other soil-atmosphere measurements (e.g. ecosystem respiration, chamber-measured net ecosystem exchange, methane fluxes) as well as experimental treatments (heterotrophic only, etc.). We give brief examples of the types of analyses possible using this new community resource and describe its accompanying R software package.

Transpiration in humid tropical forests modulates the global water cycle and is a key driver of climate regulation. Yet, our understanding of how tropical trees regulate sap flux in response to climate variability remains elusive. With a progressively warming climate, atmospheric evaporative demand [i.e., vapor pressure deficit (VPD)] will be increasingly important for plant functioning, becoming the major control of plant water use in the twenty-first century. Using measurements in 34 tree species at seven sites across a precipitation gradient in the neotropics, we determined how the maximum sap flux velocity (vmax) and the VPD threshold at which vmax is reached (VPDmax) vary with precipitation regime [mean annual precipitation (MAP); seasonal drought intensity (PDRY)] and two functional traits related to foliar and wood economics spectra [leaf mass per area (LMA); wood specific gravity (WSG)]. We show that, even though vmax is highly variable within sites, it follows a negative trend in response to increasing MAP and PDRY across sites. LMA and WSG exerted little effect on vmax and VPDmax, suggesting that these widely used functional traits provide limited explanatory power of dynamic plant responses to environmental variation within hyper-diverse forests. This study demonstrates that long-term precipitation plays an important role in the sap flux response of humid tropical forests to VPD. Our findings suggest that under higher evaporative demand, trees growing in wetter environments in humid tropical regions may be subjected to reduced water exchange with the atmosphere relative to trees growing in drier climates.

Current climate change scenarios indicate warmer temperatures and the potential for more extreme droughts in the tropics, such that a mechanistic understanding of the water cycle from individual trees to landscapes is needed to adequately predict future changes in forest structure and function. In this study, we contrasted physiological responses of tropical trees during a normal dry season with the extreme dry season due to the 2015-2016 El Niño-Southern Oscillation (ENSO) event. We quantified high resolution temporal dynamics of sap velocity (Vs), stomatal conductance (gs) and leaf water potential (ΨL) of multiple canopy trees, and their correlations with leaf temperature (Tleaf) and environmental conditions [direct solar radiation, air temperature (Tair) and vapor pressure deficit (VPD)]. The experiment leveraged canopy access towers to measure adjacent trees at the ZF2 and Tapajós tropical forest research (near the cities of Manaus and Santarém). The temporal difference between the peak of gs (late morning) and the peak of VPD (early afternoon) is one of the major regulators of sap velocity hysteresis patterns. Sap velocity displayed species-specific diurnal hysteresis patterns reflected by changes in Tleaf. In the morning, Tleaf and sap velocity displayed a sigmoidal relationship. In the afternoon, stomatal conductance declined as Tleaf approached a daily peak, allowing ΨL to begin recovery, while sap velocity declined with an exponential relationship with Tleaf. In Manaus, hysteresis indices of the variables Tleaf-Tair and ΨL-Tleaf were calculated for different species and a significant difference (p < 0.01, α = 0.05) was observed when the 2015 dry season (ENSO period) was compared with the 2017 dry season ("control scenario"). In some days during the 2015 ENSO event, Tleaf approached 40°C for all studied species and the differences between Tleaf and Tair reached as high at 8°C (average difference: 1.65 ± 1.07°C). Generally, Tleaf was higher than Tair during the middle morning to early afternoon, and lower than Tair during the early morning, late afternoon and night. Our results support the hypothesis that partial stomatal closure allows for a recovery in ΨL during the afternoon period giving an observed counterclockwise hysteresis pattern between ΨL and Tleaf.

The U.S. Department of Energy's Watershed Function Scientific Focus Area (SFA), centered in the East River, Colorado, generates diverse datasets including hydrological, geological, geochemical, geophysical, ecological, microbiological and remote sensing data. The project has deployed extensive field infrastructure involving hundreds of sensors that measure highly diverse phenomena (e.g. stream and groundwater hydrology, water quality, soil moisture, weather) across the watershed. Data from the sensor network are telemetered and automatically ingested into a queryable database. The data are subsequently quality checked, integrated with the United States Geological Survey's stream monitoring network using a custom data integration broker, and published to a portal with interactive visualizations. The resulting data products are used in a variety of scientific modeling and analytical efforts. This paper describes the SFA's end-to-end infrastructure and services that support the generation of integrated datasets from a watershed sensor network. The development and maintenance of this infrastructure, presents a suite of challenges from practical field logistics to complex data processing, which are addressed through various solutions. In particular, the SFA adopts a holistic view for data collection, assessment and integration, which dramatically improves the products generated, and enables a co-design approach wherein data collection is informed by model results and vice-versa.

Metadata describe the ancillary information needed for data preservation and independent interpretation, comparison across heterogeneous datasets, and quality assessment and quality control (QA/QC). Environmental observations are vastly diverse in type and structure, can be taken across a wide range of spatiotemporal scales in a variety of measurement settings and approaches, and saved in multiple formats. Thus, well-organized, consistent metadata are required to produce usable data products from diverse environmental observations collected across field sites. However, existing metadata reporting protocols do not support the complex data synthesis and model-data integration needs of interdisciplinary earth system research. We developed a metadata reporting framework (FRAMES) to enable management and synthesis of observational data that are essential in advancing a predictive understanding of earth systems. FRAMES utilizes best practices for data and metadata organization enabling consistent data reporting and compatibility with a variety of standardized data protocols. We used an iterative scientist-centered design process to develop FRAMES, resulting in a data reporting format that incorporates existing field practices to maximize data-entry efficiency. Thus, FRAMES has a modular organization that streamlines metadata reporting and can be expanded to incorporate additional data types. With FRAMES's multi-scale measurement position hierarchy, data can be reported at observed spatial resolutions and then easily aggregated and linked across measurement types to support model-data integration. FRAMES is in early use by both data originators (persons generating data) and consumers (persons using data and metadata). In this paper, we describe FRAMES, identify lessons learned, and discuss areas of future development.