Search

© The Author(s), 2021. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Goldstein, E. B., Buscombe, D., Lazarus, E. D., Mohanty, S. D., Rafique, S. N., Anarde, K. A., Ashton, A. D., Beuzen, T., Castagno, K. A., Cohn, N., Conlin, M. P., Ellenson, A., Gillen, M., Hovenga, P. A., Over, J.-S. R., Palermo, R., Ratliff, K. M., Reeves, I. R. B., Sanborn, L. H., Straub, J. A., Taylor, L. A., Wallace E. J., Warrick, J., Wernette, P., Williams, H. E. Labeling poststorm coastal imagery for machine learning: measurement of interrater agreement. Earth and Space Science, 8(9), (2021): e2021EA001896, https://doi.org/10.1029/2021EA001896., Classifying images using supervised machine learning (ML) relies on labeled training data—classes or text descriptions, for example, associated with each image. Data-driven models are only as good as the data used for training, and this points to the importance of high-quality labeled data for developing a ML model that has predictive skill. Labeling data is typically a time-consuming, manual process. Here, we investigate the process of labeling data, with a specific focus on coastal aerial imagery captured in the wake of hurricanes that affected the Atlantic and Gulf Coasts of the United States. The imagery data set is a rich observational record of storm impacts and coastal change, but the imagery requires labeling to render that information accessible. We created an online interface that served labelers a stream of images and a fixed set of questions. A total of 1,600 images were labeled by at least two or as many as seven coastal scientists. We used the resulting data set to investigate interrater agreement: the extent to which labelers labeled each image similarly. Interrater agreement scores, assessed with percent agreement and Krippendorff's alpha, are higher when the questions posed to labelers are relatively simple, when the labelers are provided with a user manual, and when images are smaller. Experiments in interrater agreement point toward the benefit of multiple labelers for understanding the uncertainty in labeling data for machine learning research., The authors gratefully acknowledge support from the U.S. Geological Survey (G20AC00403 to EBG and SDM), NSF (1953412 to EBG and SDM; 1939954 to EBG), Microsoft AI for Earth (to EBG and SDM), The Leverhulme Trust (RPG-2018-282 to EDL and EBG), and an Early Career Research Fellowship from the Gulf Research Program of the National Academies of Sciences, Engineering, and Medicine (to EBG). U.S. Geological Survey researchers (DB, J-SRO, JW, and PW) were supported by the U.S. Geological Survey Coastal and Marine Hazards and Resources Program as part of the response and recovery efforts under congressional appropriations through the Additional Supplemental Appropriations for Disaster Relief Act, 2019 (Public Law 116-20; 133 Stat. 871).

The ability to understand and predict storm erosion along sandy coastlines is important for safely and sustainably managing these coastlines now and in the future. This thesis investigates the use of data-driven Bayesian networks to assist coastal scientists, engineers and managers to better understand and predict sandy coastline storm erosion using large observational datasets acquired by coastal remote sensing technologies.A methodological study was undertaken using a 10-year dataset of coastal storm events extracted from Argus coastal imaging technology. It was found that Bayesian networks have key advantages for modelling storm erosion including the illumination of causality, uncertainty quantification, and low computational cost. However, these advantages can be limited by the data requirements of the approach. It was determined that, based on data availability and the modelling objective, Bayesian networks can be applied to descriptive or predictive applications. Descriptive networks can be useful for exploring the physical relationships between variables within a specific dataset. In contrast, predictive networks identify general relationships in a dataset that can be used to predict unseen data.A descriptive Bayesian network was used to investigate controls of spatial variability in storm erosion of the berm and dune based on detailed storm erosion observations spanning a 400 km region of the southeast Australian coastline. It was found that spatial variability in storm erosion was driven by both the antecedent morphology of the coastline and hydrodynamic forcing conditions of the storm event, and that controls of erosion of the berm and dune were different.A predictive Bayesian network was developed using multiple large storm erosion datasets from three different global locations. Results showed that while Bayesian networks developed at one specific location poorly predicted the storm erosion response at differing locations, a Bayesian network developed on