Search

Citizen science data are increasingly making valuable contributions to ecological studies. However, many citizen science surveys are also designed to encourage wide participation and therefore the participants have a range of natural history expertise, leading to variation and potentially bias in the data. We assessed a recently proposed measure of observer expertise, calculated based on the average numbers of species recorded by observers. We investigated if this observer expertise score is associated with how often an observer reports any individual species. Species reporting rates increased monotonically with the observer's expertise score for 197 of 200 species, suggesting that this expertise score describes inter-observer variation in the detectability of individual species. Expertise scores were incorporated into single-species occupancy models as a covariate, to explain inter-observer variation in detectability. Including expertise as a detectability covariate led to improved model fit and improved predictive performance on validation data. The expertise score had a large effect on the estimated detectability, comparable in magnitude to the effect of the duration of the observation period. Expertise scores were also included into single-species occupancy models that estimated seasonal patterns in species occupancy and seasonal expertise effects. The addition of a seasonal effect of expertise led to improved model fit and increased predictive performance on validation data. The seasonal expertise accounted for bias that may be introduced by seasonal differences in the effect of expertise, caused by changes in the environment or species behaviour. Measures of observer expertise included in models as a covariate can account for heterogeneity and bias introduced by variable expertise, although in this example the differences in estimated occupancy were small. This method of incorporating observer expertise can be used in any regression model of species occurrence, occupancy, abundance, or density to produce more reliable ecological inference and may be most important where citizen science schemes encourage wide participation. Overall, the results highlight the value of recording observer identity and other detectability covariates, to control for sources of bias associated with the observation process.

Summary 1. Species monitoring is an essential component of assessing conservation status, predicting effects of habitat change and establishing management and conservation priorities. The pervasive access to the Internet has led to the development of several extensive monitoring projects that engage massive networks of volunteers who provide observations following relatively unstructured protocols. However, the value of these data is largely unknown. 2. We develop a novel cross-data validation method for measuring the value of survey data from one source (e.g. an Internet checklist program) relative to a second, benchmark data source. The method fits a model to the data of interest and validates the model using benchmark data, allowing us to isolate the training data's information content from its biases. We also define a data efficiency ratio to quantify the relative efficiency of the data sources. 3. We apply our cross-data validation method to quantify the value of data collected in eBird – a western hemisphere, year-round citizen science bird checklist project – relative to data from the highly standardized North American Breeding Bird Survey (BBS). The results show that eBird data contain information similar in quality to that in BBS data, while the information per BBS datum is higher. 4. We suggest that these methods have more general use in evaluating the suitability of sources of data for addressing specific questions for taxa of interest.