Good practice guidelines, protocols and benchmarking standards for quality assurance

Citizen observatories offer a potential advancement in the field of Land Use and Land Cover (LULC) monitoring, greatly increasing the capacity and frequency of data collection, enabling, for example, near-real-time response to emerging environmental hazards (e.g., deforestation, flooding). However, concerns have been raised about the quality of volunteered geographic information (VGI) when compared to the traditional land surveying techniques and methods employed by national mapping agencies. A key element of the LandSense citizen observatory, described in this report, was to provide robust quality assurance tools and methods to ensure that the quality of VGI data can be characterised and made effective for user needs. This final deliverable (D5.7) from the Quality Assurance (QA) work package of the LandSense project builds upon the findings to-date and focuses on general good practice guidance for data protocols and QA services. It also includes the results of benchmarking LULC data collected by LandSense contributors against national authoritative LULC datasets. The design of data models and protocols for their use in citizen observatories are discussed, emphasising the need to balance consistency and flexibility across a diverse set of pilot studies. Evolution of data design models varied greatly across the pilot studies, and it is evident that QA services should be designed to allow for changes to the underlying data model while minimising disruption to the design of the QA services. The four key concepts to be considered in forming a robust data model from a QA perspective (Adaptability, Version control, Accessibility and Data Protection) are described. To evaluate the potential value and accuracy of crowdsourced LULC data, the OSMlanduse1 product was compared with authoritative mapping datasets for three EU countries included in the LandSense pilot studies (Germany, France and Austria). The results of this benchmarking exercise showed strong agreement in the majority of land use categories when compared to the EU CORINE Land Cover (CLC) classification system. Agreement was less strong when compared to National Mapping Agency (NMA) products. Distinguishing between different types of built up areas (e.g., between urban fabric and commercial/industrial land use) and specific agricultural land uses were found to be areas where differences were most commonly identified across all three countries. Good practice guidelines for each of the high-level QA services developed as part of the LandSense project form the heart of this report. These include checks on polygonal data, image quality and privacy checks, positional accuracy and offset, contributor agreement and categorical accuracy. For each service, QA issues relating to its implementation are described, such as: how success/failure may be defined, dealing with borderline cases, examples of both good and poor practice and potential methods identified for improvement.