Search

This deliverable reports on the use of the quality assurance (QA) service, developed as part of WP5, to analyse data collected by the LandSense demonstrator pilot case studies. It describes data acquired by the pilots and the preliminary analysis that results from processing the data using the QA service. Assessments of data quality relating to user-captured photographs, ensuring GDPR compliance on user captured photographs, assessing point and polygon field observations, calculating categorical accuracy of classifications of Earth Observation data, and contributor agreement measures are described.

Citizen observations have the potential to revolutionise the field of Land Use and Land Cover (LULC) monitoring, greatly increasing reporting capacity and enabling near real-time response to emerging environmental hazards (e.g., deforestation, flooding). However, accuracy and other data quality issues are a key concern when utilising citizen observations. A suite of Quality Assurance (QA) tools suitable for LandSense were identified (D5.1) and implemented in phase I of the project (D5.4). This deliverable reports on the wider implementation of all QA tools with the data collected in phases I and II of the project and provides detailed results from this work. The use and performance of eight QA tools is discussed across all three LandSense themes (urban landscape dynamics, agricultural land use and forest and habitat monitoring) using heterogeneous datasets from six pilot studies. The QA platform performed as designed and no notable operational errors were encountered. Modifications and additions were made to some of the QA tools in light of the findings of D5.4 and are described here. Quality and privacy checks on photographic data collected performed well (e.g. 90%+ accuracy in detecting privacy features) and correlations between photo quality and feature detection were investigated and described. Image blur was not found to be a significant problem and only detected in specific instances (i.e. low light conditions and images taken from moving vehicles). QA checks were used to assess hundreds of user observations of LULC features and demonstrated the ability to identify areas of both high and low agreement between multiple contributors. Links between contributor agreement and the type of LULC feature are also described. Building on this work, QA checks on the categorical accuracy of user contributions were performed and found to be very promising. It was found that Volunteered Geographic Information (VGI) was of sufficiently good quality for identifying key types of LULC, such as residential land use change or detecting and identifying the urban fabric. However, some specific LULC features were harder for VGI to identify accurately, e.g., distinguishing between different types of agricultural land use. The results outlined in this deliverable will form the basis for development of the LandSense QA good practice guide (D5.7).

Citizen observatories offer a potential advancement in the field of Land Use and Land Cover (LULC) monitoring, greatly increasing the capacity and frequency of data collection, enabling, for example, near-real-time response to emerging environmental hazards (e.g., deforestation, flooding). However, concerns have been raised about the quality of volunteered geographic information (VGI) when compared to the traditional land surveying techniques and methods employed by national mapping agencies. A key element of the LandSense citizen observatory, described in this report, was to provide robust quality assurance tools and methods to ensure that the quality of VGI data can be characterised and made effective for user needs. This final deliverable (D5.7) from the Quality Assurance (QA) work package of the LandSense project builds upon the findings to-date and focuses on general good practice guidance for data protocols and QA services. It also includes the results of benchmarking LULC data collected by LandSense contributors against national authoritative LULC datasets. The design of data models and protocols for their use in citizen observatories are discussed, emphasising the need to balance consistency and flexibility across a diverse set of pilot studies. Evolution of data design models varied greatly across the pilot studies, and it is evident that QA services should be designed to allow for changes to the underlying data model while minimising disruption to the design of the QA services. The four key concepts to be considered in forming a robust data model from a QA perspective (Adaptability, Version control, Accessibility and Data Protection) are described. To evaluate the potential value and accuracy of crowdsourced LULC data, the OSMlanduse1 product was compared with authoritative mapping datasets for three EU countries included in the LandSense pilot studies (Germany, France and Austria). The results of this benchmarking exercise showed strong agreement in the majority of land use categories when compared to the EU CORINE Land Cover (CLC) classification system. Agreement was less strong when compared to National Mapping Agency (NMA) products. Distinguishing between different types of built up areas (e.g., between urban fabric and commercial/industrial land use) and specific agricultural land uses were found to be areas where differences were most commonly identified across all three countries. Good practice guidelines for each of the high-level QA services developed as part of the LandSense project form the heart of this report. These include checks on polygonal data, image quality and privacy checks, positional accuracy and offset, contributor agreement and categorical accuracy. For each service, QA issues relating to its implementation are described, such as: how success/failure may be defined, dealing with borderline cases, examples of both good and poor practice and potential methods identified for improvement.&nbsp

Citizen observatories offer a potential advancement in the field of Land Use and Land Cover (LULC) monitoring, greatly increasing the capacity and frequency of data collection, enabling, for example, near-real-time response to emerging environmental hazards (e.g., deforestation, flooding). However, concerns have been raised about the quality of volunteered geographic information (VGI) when compared to the traditional land surveying techniques and methods employed by national mapping agencies. A key element of the LandSense citizen observatory, described in this report, was to provide robust quality assurance tools and methods to ensure that the quality of VGI data can be characterised and made effective for user needs. This final deliverable (D5.7) from the Quality Assurance (QA) work package of the LandSense project builds upon the findings to-date and focuses on general good practice guidance for data protocols and QA services. It also includes the results of benchmarking LULC data collected by LandSense contributors against national authoritative LULC datasets. The design of data models and protocols for their use in citizen observatories are discussed, emphasising the need to balance consistency and flexibility across a diverse set of pilot studies. Evolution of data design models varied greatly across the pilot studies, and it is evident that QA services should be designed to allow for changes to the underlying data model while minimising disruption to the design of the QA services. The four key concepts to be considered in forming a robust data model from a QA perspective (Adaptability, Version control, Accessibility and Data Protection) are described. To evaluate the potential value and accuracy of crowdsourced LULC data, the OSMlanduse1 product was compared with authoritative mapping datasets for three EU countries included in the LandSense pilot studies (Germany, France and Austria). The results of this benchmarking exercise showed strong agreement in the majority of land use categories when compared to the EU CORINE Land Cover (CLC) classification system. Agreement was less strong when compared to National Mapping Agency (NMA) products. Distinguishing between different types of built up areas (e.g., between urban fabric and commercial/industrial land use) and specific agricultural land uses were found to be areas where differences were most commonly identified across all three countries. Good practice guidelines for each of the high-level QA services developed as part of the LandSense project form the heart of this report. These include checks on polygonal data, image quality and privacy checks, positional accuracy and offset, contributor agreement and categorical accuracy. For each service, QA issues relating to its implementation are described, such as: how success/failure may be defined, dealing with borderline cases, examples of both good and poor practice and potential methods identified for improvement.

Citizen observations have the potential to revolutionise the field of Land Use and Land Cover (LULC) monitoring, greatly increasing reporting capacity and enabling near real-time response to emerging environmental hazards (e.g., deforestation, flooding). However, accuracy and other data quality issues are a key concern when utilising citizen observations. A suite of Quality Assurance (QA) tools suitable for LandSense were identified (D5.1) and implemented in phase I of the project (D5.4). This deliverable reports on the wider implementation of all QA tools with the data collected in phases I and II of the project and provides detailed results from this work. The use and performance of eight QA tools is discussed across all three LandSense themes (urban landscape dynamics, agricultural land use and forest and habitat monitoring) using heterogeneous datasets from six pilot studies. The QA platform performed as designed and no notable operational errors were encountered. Modifications and additions were made to some of the QA tools in light of the findings of D5.4 and are described here. Quality and privacy checks on photographic data collected performed well (e.g. 90%+ accuracy in detecting privacy features) and correlations between photo quality and feature detection were investigated and described. Image blur was not found to be a significant problem and only detected in specific instances (i.e. low light conditions and images taken from moving vehicles). QA checks were used to assess hundreds of user observations of LULC features and demonstrated the ability to identify areas of both high and low agreement between multiple contributors. Links between contributor agreement and the type of LULC feature are also described. Building on this work, QA checks on the categorical accuracy of user contributions were performed and found to be very promising. It was found that Volunteered Geographic Information (VGI) was of sufficiently good quality for identifying key types of LULC, such as residential land use change or detecting and identifying the urban fabric. However, some specific LULC features were harder for VGI to identify accurately, e.g., distinguishing between different types of agricultural land use. The results outlined in this deliverable will form the basis for development of the LandSense QA good practice guide (D5.7).