Search

Measuring forest biodiversity using terrestrial surveys is expensive and can only capture common species abundance in large heterogeneous landscapes. In contrast, combining airborne imagery with computer vision can generate individual tree data at the scales of hundreds of thousands of trees. To train computer vision models, ground-based species labels are combined with airborne reflectance data. Due to the difficulty of finding rare species in a large landscape, the majority of classification models only include the most abundant species, leading to biased predictions at broad scales. Extending classification models to include rare species requires targeted data collection and algorithmic improvements to overcome large data imbalances between dominant and rare taxa. In addition, large landscapes often require multiple acquisition events, leading to significant within-species variation in reflectance spectra. Using a multi-temporal hierarchical model, we demonstrate the ability to include species predicted at less than 1% frequency in landscape without losing performance on the dominant species. The final model has over 75% accuracy for 14 species with improved rare species classification compared to a baseline deep learning model. After filtering out dead trees, we generate landscape species maps of individual crowns for over 670,000 individual trees at the Ordway Swisher Biological Station within the National Ecological Observatory Network. We estimate the relative abundance of the species within the landscape and provide three measures of uncertainty to generate a range of counts for each species. These maps provide the first estimates of canopy tree diversity within NEON sites to include rare species and provide a blueprint for capturing tree diversity using airborne computer vision at broad scales.

Advances in artificial intelligence for computer vision hold great promise for increasing the scales at which ecological systems can be studied. The distribution and behavior of individuals is central to ecology, and computer vision using deep neural networks can learn to detect individual objects in imagery. However, developing supervised models for ecological monitoring is challenging because it requires large amounts of human-labeled training data, requires advanced technical expertise and computational infrastructure, and is prone to overfitting. This limits application across space and time. One solution is developing generalized models that can be applied across species and ecosystems. Using over 250,000 annotations from 13 projects from around the world, we develop a general bird detection model that achieves over 65% recall and 50% precision on novel aerial data without any local training despite differences in species, habitat, and imaging methodology. Fine-tuning this model with only 1000 local annotations increases these values to an average of 84% recall and 69% precision by building on the general features learned from other data sources. Retraining from the general model improves local predictions even when moderately large annotation sets are available and makes model training faster and more stable. Our results demonstrate that general models for detecting broad classes of organisms using airborne imagery are achievable. These models can reduce the effort, expertise, and computational resources necessary for automating the detection of individual organisms across large scales, helping to transform the scale of data collection in ecology and the questions that can be addressed.

Advances in remote sensing imagery and machine learning applications unlock the potential for developing algorithms for species classification at the level of individual tree crowns at unprecedented scales. However, most approaches to date focus on site-specific applications and a small number of taxonomic groups. Little is known about how well these approaches generalize across broader geographic areas and ecosystems. Leveraging field surveys and hyperspectral remote sensing data from the National Ecological Observatory Network (NEON), we developed a continental-extent model for tree species classification that can be applied to the network, including a wide range of US terrestrial ecosystems. We compared the performance of a model trained with data from 27 NEON sites to models trained with data from each individual site, evaluating advantages and challenges posed by training species classifiers at the US scale. We evaluated the effect of geographic location, topography, and ecological conditions on the accuracy and precision of species predictions (72 out of 77 species). On average, the general model resulted in good overall classification accuracy (micro-F1 score), with better accuracy than site-specific classifiers (average individual tree level accuracy of 0.77 for the general model and 0.70 for site-specific models). Aggregating species to the genus-level increased accuracy to 0.83. Regions with more species exhibited lower classification accuracy. Predicted species were more likely to be confused with congeneric and co-occurring species and confusion was highest for trees with structural damage and in complex closed-canopy forests. The model produced accurate estimates of uncertainty, correctly identifying trees where confusion was likely. Using only data from NEON, this single integrated classifier can make predictions for 20% of all tree species found in forest ecosystems across the entire US, which make up to roughly 90% of the upper canopy of the studied ecosystems. This suggests the potential for integrating information from multiple datasets and locations to develop broad scale general models for species classification from hyperspectral imaging.

Scene understanding methodologies (e.g., classification, segmentation, and anomaly detection) can be costly when operating on a per-pixel basis. As remote sensing applications rely on imagery with ...

We construct infinitesimally rigid bipartite unit-bar frameworks in $$\mathbb {R}^d$$ . Our constructions are variants of the knight’s graph. This answers problems proposed by Maehara.

Delineating and classifying individual trees in remote sensing data is challenging. Many tree crown delineation methods have difficulty in closed-canopy forests and do not leverage multiple datasets. Methods to classify individual species are often accurate for common species, but perform poorly for less common species and when applied to new sites. We ran a data science competition to help identify effective methods for delineation of individual crowns and classification to determine species identity. This competition included data from multiple sites to assess the methods’ ability to generalize learning across multiple sites simultaneously, and transfer learning to novel sites where the methods were not trained. Six teams, representing 4 countries and 9 individual participants, submitted predictions. Methods from a previous competition were also applied and used as the baseline to understand whether the methods are changing and improving over time. The best delineation method was based on an instance segmentation pipeline, closely followed by a Faster R-CNN pipeline, both of which outperformed the baseline method. However, the baseline (based on a growing region algorithm) still performed well as did the Faster R-CNN. All delineation methods generalized well and transferred to novel forests effectively. The best species classification method was based on a two-stage fully connected neural network, which significantly outperformed the baseline (a random forest and Gradient boosting ensemble). The classification methods generalized well, with all teams training their models using multiple sites simultaneously, but the predictions from these trained models generally failed to transfer effectively to a novel site. Classification performance was strongly influenced by the number of field-based species IDs available for training the models, with most methods predicting common species well at the training sites. Classification errors (i.e., species misidentification) were most common between similar species in the same genus and different species that occur in the same habitat. The best methods handled class imbalance well and learned unique spectral features even with limited data. Most methods performed better than baseline in detecting new (untrained) species, especially in the site with no training data. Our experience further shows that data science competitions are useful for comparing different methods through the use of a standardized dataset and set of evaluation criteria, which highlights promising approaches and common challenges, and therefore advances the ecological and remote sensing field as a whole.

Advances in artificial intelligence for computer vision hold great promise for increasing the scales at which ecological systems can be studied. The distribution and behavior of individuals is central to ecology, and computer vision using deep neural networks can learn to detect individual objects in imagery. However, developing supervised models for ecological monitoring is challenging because it needs large amounts of human-labeled training data, requires advanced technical expertise and computational infrastructure, and is prone to overfitting. This limits application across space and time. One solution is developing generalized models that can be applied across species and ecosystems. Using over 250,000 annotations from 13 projects from around the world, we develop a general bird detection model that achieves over 65% recall and 50% precision on novel aerial data without any local training despite differences in species, habitat, and imaging methodology. Fine-tuning this model with only 1000 local annotations increase these values to an average of 84% recall and 69% precision by building on the general features learned from other data sources. Retraining from the general model improves local predictions even when moderately large annotation sets are available and makes model training faster and more stable. Our results demonstrate that general models for detecting broad classes of organisms using airborne imagery are achievable. These models can reduce the effort, expertise, and computational resources necessary for automating the detection of individual organisms across large scales, helping to transform the scale of data collection in ecology and the questions that can be addressed.

Broad scale remote sensing promises to build forest inventories at unprecedented scales. A crucial step in this process is to associate sensor data into individual crowns. While dozens of crown detection algorithms have been proposed, their performance is typically not compared based on standard data or evaluation metrics. There is a need for a benchmark dataset to minimize differences in reported results as well as support evaluation of algorithms across a broad range of forest types. Combining RGB, LiDAR and hyperspectral sensor data from the USA National Ecological Observatory Network’s Airborne Observation Platform with multiple types of evaluation data, we created a benchmark dataset to assess crown detection and delineation methods for canopy trees covering dominant forest types in the United States. This benchmark dataset includes an R package to standardize evaluation metrics and simplify comparisons between methods. The benchmark dataset contains over 6,000 image-annotated crowns, 400 field-annotated crowns, and 3,000 canopy stem points from a wide range of forest types. In addition, we include over 10,000 training crowns for optional use. We discuss the different evaluation data sources and assess the accuracy of the image-annotated crowns by comparing annotations among multiple annotators as well as overlapping field-annotated crowns. We provide an example submission and score for an open-source algorithm that can serve as a baseline for future methods., Author summary Combining RGB, LiDAR and hyperspectral sensor data from the USA National Ecological Observatory Network’s Airborne Observation Platform with multiple types of evaluation data, we created a benchmark dataset to assess crown detection and delineation methods for canopy trees covering dominant forest types in the United States. This benchmark dataset includes an R package to standardize evaluation metrics and simplify comparisons between methods. The benchmark dataset contains over 6,000 image-annotated crowns, 400 field-annotated crowns, and 3,000 canopy stem points from a wide range of forest types.

Supervised methods for object delineation in remote sensing require labeled ground-truth data. Gathering sufficient high quality ground-truth data is difficult, especially when targets are of irregular shape or difficult to distinguish from background or neighboring objects. Tree crown delineation provides key information from remote sensing images for forestry, ecology, and management. However, tree crowns in remote sensing imagery are often difficult to label and annotate due to irregular shape, overlapping canopies, shadowing, and indistinct edges. There are also multiple approaches to annotation in this field (e.g., rectangular boxes vs. convex polygons) that further contribute to annotation imprecision. However, current evaluation methods do not account for this uncertainty in annotations, and quantitative metrics for evaluation can vary across multiple annotators. In this paper, we address these limitations by developing an adaptation of the Rand index for weakly-labeled crown delineation that we call RandCrowns. Our new RandCrowns evaluation metric provides a method to appropriately evaluate delineated tree crowns while taking into account imprecision in the ground-truth delineations. The RandCrowns metric reformulates the Rand index by adjusting the areas over which each term of the index is computed to account for uncertain and imprecise object delineation labels. Quantitative comparisons to the commonly used intersection over union method shows a decrease in the variance generated by differences among multiple annotators. Combined with qualitative examples, our results suggest that the RandCrowns metric is more robust for scoring target delineations in the presence of uncertainty and imprecision in annotations that are inherent to tree crown delineation.

Functional traits are central to how organisms perform and influence ecosystem function. Although phylogenetic constraints and environmental conditions are both known to affect trait distributions, data limitations have resulted in large scale studies modeling traits either as species weighted averages (ignoring intraspecific variation) or as a function of the environment (ignoring phylogenetic constraints). As a result, large scale predictions for trait distributions do not include key drivers, likely resulting in biased predictions, and cannot be used to assess the relative contributions of inter- and intraspecific variation. To address these limitations, we developed a joint model integrating phylogenetic and environmental information to understand and predict the distribution of eight leaf traits across the eastern United States. This joint model explained 68% of trait variation, outperforming both species-only and environment-only models, with variance attributable to phylogeny alone (23%), the environment alone (18%), and their overlapping effects (26%). The importance of phylogenetic constraints and the environment varied by trait, with some traits associated predominantly with environmental variation and others with phylogeny. To make predictions more continuously across the eastern USA we combined this model with data from the large-scale Forest Inventory and Analysis survey to estimate traits for ~1.2 million trees. The combined model exhibited significant deviations in predictions from both species-only and environment-only models with variation in the direction and magnitude of these differences among ecoregions. These predictions demonstrate the importance of modeling both intra- and interspecific variation to understand and predict large scale gradients in species and ecosystem traits. Significance Statement Large scale trait studies typically rely on species level average trait values, but recent research suggests that intra-species variation plays a key role in the ecology of species across their range of distribution. We developed a new continental scale approach that integrates inter- and intra-specific variation on leaf traits by leveraging the interaction between the environment and the phylogenetic relationships across species. Using this approach to make predictions for over a million trees covering the Eastern United States shows that combining intra- and inter-species variation is crucial for capturing patterns at the continental scale. This approach represents a novel state of the art method with the potential to become the new standard for understanding traits at large scales.

Probabilistic near-term forecasting facilitates evaluation of model predictions against observations and is of pressing need in ecology to inform environmental decision making and effect societal change. Despite this imperative, many ecologists are unfamiliar with the widely used tools for evaluating probabilistic forecasts developed in other fields. We address this gap by reviewing the literature on probabilistic forecast evaluation from diverse fields including climatology, economics, and epidemiology. We present established practices for selecting evaluation data (end-sample hold out), graphical forecast evaluation (times-series plots with uncertainty, Probability Integral Transform plots), quantitative evaluation using scoring rules (log, quadratic, spherical, and ranked probability scores), and comparing scores across models (skill score, Diebold-Mariano test). We cover common approaches, highlight mathematical concepts to follow, and note decision points to allow application of general principles to specific forecasting endeavors. We illustrate these approaches with an application to a long-term rodent population time series currently used for ecological forecasting and discuss how ecology can continue to learn from and drive the cross-disciplinary field of forecasting science.

Broad scale remote sensing promises to build forest inventories at unprecedented scales. A crucial step in this process is designing individual tree segmentation algorithms to associate pixels into delineated tree crowns. While dozens of tree delineation algorithms have been proposed, their performance is typically not compared based on standard data or evaluation metrics, making it difficult to understand which algorithms perform best under what circumstances. There is a need for an open evaluation benchmark to minimize differences in reported results due to data quality, forest type and evaluation metrics, and to support evaluation of algorithms across a broad range of forest types. Combining RGB, LiDAR and hyperspectral sensor data from the National Ecological Observatory Network’s Airborne Observation Platform with multiple types of evaluation data, we created a novel benchmark dataset to assess individual tree delineation methods. This benchmark dataset includes an R package to standardize evaluation metrics and simplify comparisons between methods. The benchmark dataset contains over 6,000 image-annotated crowns, 424 field-annotated crowns, and 3,777 overstory stem points from a wide range of forest types. In addition, we include over 10,000 training crowns for optional use. We discuss the different evaluation sources and assess the accuracy of the image-annotated crowns by comparing annotations among multiple annotators as well as to overlapping field-annotated crowns. We provide an example submission and score for an open-source baseline for future methods.

Forests provide essential biodiversity, ecosystem and economic services. Information on individual trees is important for understanding the state of forest ecosystems but obtaining individual-level data at broad scales is challenging due to the costs and logistics of data collection. While advances in remote sensing techniques allow surveys of individual trees at unprecedented extents, there remain significant technical and computational challenges in turning sensor data into tangible information. Using deep learning methods, we produced an open-source dataset of individual-level crown estimates for 100 million trees at 37 sites across the United States surveyed by the National Ecological Observatory Network’s Airborne Observation Platform. Each canopy tree crown is represented by a rectangular bounding box and includes information on the height, crown area, and spatial location of the tree. Tree crowns identified using this technique correspond well with hand-labeled crowns, exhibiting both high levels of overlap and good correspondence in height estimates. These data have the potential to drive significant expansion of individual-level research on trees by facilitating both regional analyses at scales of ~10,000 ha and cross-region comparisons encompassing forest types from most of the United States.

Evaluation of ecological forecasts is a vital step in the continuous improvement of near-term ecological forecasts. Here we performed a thorough evaluation of a near-term phenological forecast system which has been operating for several years. We evaluated point forecast accuracy and the reliability of the prediction intervals. We also tested the contribution of upstream climate forecasts on phenology forecast proficiency. We found that 9 month climate forecasts contributed little skill overall, though some species did benefit from them. The assimilation of observed winter and spring temperature provided the largest improvement of forecast skill throughout the spring. We also found that phenology forecast prediction intervals were most robust when uncertainty was propagated from climate, phenological model, and model parameters as opposed to using climate uncertainty alone. Our analysis points the way toward several potential improvements to the forecasting system, which can be re-evaluated at a future date in a continuous cycle of forecast refinement.

Functional ecology has increasingly focused on describing ecological communities based on their traits (measurable features affecting individuals' fitness and performance). Analyzing trait distributions within and among forests could significantly improve understanding of community composition and ecosystem function. Historically, data on trait distributions are generated by (1) collecting a small number of leaves from a small number of trees, which suffers from limited sampling but produces information at the fundamental ecological unit (the individual), or (2) using remote-sensing images to infer traits, producing information continuously across large regions, but as plots (containing multiple trees of different species) or pixels, not individuals. Remote-sensing methods that identify individual trees and estimate their traits would provide the benefits of both approaches, producing continuous large-scale data linked to biological individuals. We used data from the National Ecological Observatory Network (NEON) to develop a method to scale up functional traits from 160 trees to the millions of trees within the spatial extent of two NEON sites. The pipeline consists of three stages: (1) image segmentation, to identify individual trees and estimate structural traits; (2) an ensemble of models to infer leaf mass area (LMA), nitrogen, carbon, and phosphorus content using hyperspectral signatures, and DBH from allometry; and (3) predictions for segmented crowns for the full remote-sensing footprint at the NEON sites. The R2 values on held-out test data ranged from 0.41 to 0.75 on held-out test data. The ensemble approach performed better than single partial least-squares models. Carbon performed poorly compared to other traits (R2 of 0.41). The crown segmentation step contributed the most uncertainty in the pipeline, due to over-segmentation. The pipeline produced good estimates of DBH (R2 of 0.62 on held-out data). Trait predictions for crowns performed significantly better than comparable predictions on pixels, resulting in improvement of R2 on test data of between 0.07 and 0.26. We used the pipeline to produce individual-level trait data for ~5 million individual crowns, covering a total extent of ~360 km2 . This large data set allows testing ecological questions on landscape scales, revealing that foliar traits are correlated with structural traits and environmental conditions.

Remote sensing of forested landscapes can transform the speed, scale, and cost of forest research. The delineation of individual trees in remote sensing images is an essential task in forest analysis. Here we introduce a new Python package, DeepForest, that detects individual trees in high resolution RGB imagery using deep learning.While deep learning has proven highly effective in a range of computer vision tasks, it requires large amounts of training data that are typically difficult to obtain in ecological studies. DeepForest overcomes this limitation by including a model pre-trained on over 30 million algorithmically generated crowns from 22 forests and fine-tuned using 10,000 hand-labeled crowns from 6 forests.The package supports the application of this general model to new data, fine tuning the model to new datasets with user labeled crowns, training new models, and evaluating model predictions. This simplifies the process of using and retraining deep learning models for a range of forests, sensors, and spatial resolutions.We illustrate the workflow of DeepForest using data from the National Ecological Observatory Network, a tropical forest in French Guiana, and street trees from Portland, Oregon.

Two foundational questions about sustainability are “How are ecosystems and the services they provide going to change in the future?” and “How do human decisions affect these trajectories?” Answering these questions requires an ability to forecast ecological processes. Unfortunately, most ecological forecasts focus on centennial-scale climate responses, therefore neither meeting the needs of near-term (daily to decadal) environmental decision-making nor allowing comparison of specific, quantitative predictions to new observational data, one of the strongest tests of scientific theory. Near-term forecasts provide the opportunity to iteratively cycle between performing analyses and updating predictions in light of new evidence. This iterative process of gaining feedback, building experience, and correcting models and methods is critical for improving forecasts. Iterative, near-term forecasting will accelerate ecological research, make it more relevant to society, and inform sustainable decision-making under high uncertainty and adaptive management. Here, we identify the immediate scientific and societal needs, opportunities, and challenges for iterative near-term ecological forecasting. Over the past decade, data volume, variety, and accessibility have greatly increased, but challenges remain in interoperability, latency, and uncertainty quantification. Similarly, ecologists have made considerable advances in applying computational, informatic, and statistical methods, but opportunities exist for improving forecast-specific theory, methods, and cyberinfrastructure. Effective forecasting will also require changes in scientific training, culture, and institutions. The need to start forecasting is now; the time for making ecology more predictive is here, and learning by doing is the fastest route to drive the science forward.

The scale and magnitude of complex and pressing environmental issues lend urgency to the need for integrative and reproducible analysis and synthesis, facilitated by data-intensive research approaches. However, the recent pace of technological change has been such that appropriate skills to accomplish data-intensive research are lacking among environmental scientists, who more than ever need greater access to training and mentorship in computational skills. Here, we provide a roadmap for raising data competencies of current and next-generation environmental researchers by describing the concepts and skills needed for effectively engaging with the heterogeneous, distributed, and rapidly growing volumes of available data. We articulate five key skills: (1) data management and processing, (2) analysis, (3) software skills for science, (4) visualization, and (5) communication methods for collaboration and dissemination. We provide an overview of the current suite of training initiatives available to environmental scientists and models for closing the skill-transfer gap.

We prove that the number of directions contained in a set of the form $A \times B \subset AG(2,p)$, where $p$ is prime, is at least $|A||B| - \min\{|A|,|B|\} + 2$. Here $A$ and $B$ are subsets of $GF(p)$ each with at least two elements and $|A||B, Comment: 8 pages

Most phenomenological, statistical models used to generate ecological forecasts take either a time-series approach, based on long-term data from one location, or a space-for-time approach, based on data describing spatial patterns across environmental gradients. Here we consider how the forecast horizon determines whether more accurate predictions come from the time-series approach, the space-for-time approach, or a combination of the two. We use two simulated case studies to show that forecasts for short and long forecast horizons need to focus on different ecological processes, which are reflected in different kinds of data. In the short-term, dynamics reflect initial conditions and fast processes such as birth and death, and the time-series approach makes the best predictions. In the long-term, dynamics reflect the additional influence of slower processes such as evolutionary and ecological selection, colonization and extinction, which the space-for-time approach can effectively capture. At intermediate time-scales, a weighted average of the two approaches shows promise. However, making this weighted model operational will require new research to predict the rate at which slow processes begin to influence dynamics.

Tree detection is a fundamental task in remote sensing for forestry and ecosystem ecology applications. While many individual tree segmentation algorithms have been proposed, the development and testing of these algorithms is typically site specific, with few methods evaluated against data from multiple forest types simultaneously. This makes it difficult to determine the generalization of proposed approaches, and limits tree detection at broad scales. Using data from the National Ecological Observatory Network we extend a recently developed semi-supervised deep learning algorithm to include data from a range of forest types, determine whether information from one forest can be used for tree detection in other forests, and explore the potential for building a universal tree detection algorithm. We find that the deep learning approach works well for overstory tree detection across forest conditions, outperforming conventional LIDAR-only methods in all forest types. Performance was best in open oak woodlands and worst in alpine forests. When models were fit to one forest type and used to predict another, performance generally decreased, with better performance when forests were more similar in structure. However, when models were pretrained on data from other sites and then fine-tuned using a small amount of hand-labeled data from the evaluation site, they performed similarly to local site models. Most importantly, a universal model fit to data from all sites simultaneously performed as well or better than individual models trained for each local site. This result suggests that RGB tree detection models that can be applied to a wide array of forest types at broad scales should be possible.

Author summary Postdocs are a critical transition for early-career researchers. This transient period, between finishing a PhD and finding a permanent position, is when early-career researchers develop independent research programs and establish collaborative relationships that can make a successful career. Traditionally, postdocs physically relocate—sometimes multiple times—for these short-term appointments, which creates challenges that can disproportionately affect members of traditionally underrepresented groups in science, technology, engineering, and mathematics (STEM). However, many research activities involving analytical and quantitative work do not require a physical presence in a lab and can be accomplished remotely. Other fields have embraced remote work, yet many academics have been hesitant to hire remote postdocs. In this article, we present advice to both principal investigators (PIs) and postdocs for successfully navigating a remote position. Using the combined experience of the authors (as either remote postdocs or employers of remote postdocs), we provide a road map to overcome the real (and perceived) obstacles associated with remote work. With planning, communication, and creativity, remote postdocs can be a fully functioning and productive member of a research lab. Further, our rules can be useful for research labs generally and can help foster a more flexible and inclusive environment.