Search

A polarimetric synthetic aperture radar (PolSAR) sensor is able to collect images in different polarization states, making it a rich source of information for target characterization. PolSAR images are inherently affected by speckle. Therefore, before deriving ad hoc products from the data, the polarimetric covariance matrix needs to be estimated by reducing speckle. In recent years, deep learning-based despeckling methods have started to evolve from single-channel SAR images to PolSAR images. To this aim, deep learning-based approaches separate the real and imaginary components of the complex-valued covariance matrix and use them as independent channels in standard convolutional neural networks (CNNs). However, this approach neglects the mathematical relationship that exists between the real and imaginary components, resulting in suboptimal output. Here, we propose a multistream complex-valued fully convolutional network (FCN) (CV-deSpeckNet1) to reduce speckle and effectively estimate the PolSAR covariance matrix. To evaluate the performance of CV-deSpeckNet, we used Sentinel-1 dual polarimetric SAR images to compare against its real-valued counterpart that separates the real and imaginary parts of the complex covariance matrix. CV-deSpeckNet was also compared against the state of the art PolSAR despeckling methods. The results show that CV-deSpeckNet was able to be trained with a fewer number of samples, has a higher generalization capability, and resulted in higher accuracy than its real-valued counterpart and state-of-the-art PolSAR despeckling methods. These results showcase the potential of complex-valued deep learning for PolSAR despeckling.

Deep learning methods based on Fully convolution networks (FCNs) have shown an impressive progress in building outline delineation from very high resolution (VHR) remote sensing (RS) imagery. Common issues still exist in extracting precise building shapes and outlines, often resulting in irregular edges and over smoothed corners. In this paper, we use PolyMapper, a recently introduced deep-learning framework that is able to predict object outlines in a vector representation directly. We have introduced two main modifications to this baseline method. First, we introduce EffcientNet as backbone feature encoder to our network, which uses compound coefficient to scale up all dimensions of depth/width/resolution uniformly, to improve the processing speed with fewer parameters. Second, we integrate a boundary refinement block (BRB) to strengthen the boundary feature learning and to further improve the accuracy of corner prediction. The results demonstrate that the end-to-end learnable model is capable of delineating polygons of building outlines that closely approximate the structure of reference labels. Experiments on the crowdAI building instance segmentation datasets show that our model outperforms PolyMapper in all COCO metrics, for instance showing a 0.13 higher mean Average Precision (AP) value and a 0.60 higher mean Average Recall value. Also qualitative results show that our method segments building instances of various shapes more accurately.

One of the major challenges in precision viticulture in Europe is the detection and mapping of flavescence dorée (FD) grapevine disease to monitor and contain its spread. The lack of effective cures and the need for sustainable preventive measures are nowadays crucial issues. Insecticides and the plants uprooting are commonly employed to withhold disease infection, even if these solutions imply serious economic consequences and a strong environmental impact. The development of a rapid strategy to identify the disease is required to cover large portions of the crop and thus to limit damages in a time-effective way. This paper investigates the use of Unmanned Aerial Vehicles (UAVs), a cost-effective approach to early detection of diseased areas. We address this task with an object detection deep network, Faster R-CNN, instead of a traditional pixel-wise classifier. This work tests Faster R-CNN performance on this specific application through a comparative analysis with a pixel-wise classification algorithm (Random Forest). To take advantage of the full image resolution, the experimental analysis is performed using the original UAV imagery acquired in real conditions (instead of the derived orthomosaic). The first result of this paper is the definition of a new dataset for FD disease identification by UAV original imagery at the canopy scale. Moreover, we demonstrate the feasibility of applying Faster-R-CNN as a quasi-real-time alternative solution to semantic segmentation. The trained Faster-R-CNN achieved an average precision of 82% on the test set.

The application of UAV-based aerial imagery has advanced exponentially in the past two decades. This can be attributed to UAV operational flexibility, ultra-high spatial resolution, inexpensiveness, and UAV-based sensors enhancement. Nonetheless, the application of multitemporal series of multispectral UAV imagery still suffers significant misregistration errors, and therefore becoming a concern for applications such as precision agriculture. Direct image georeferencing and co-registration is commonly done using ground control points; this is usually costly and time consuming. This research proposes a novel approach for automatic co-registration of multitemporal UAV imagery using intensity-based keypoints. The Speeded Up Robust Features (SURF), Binary Robust Invariant Scalable Keypoints (BRISK), Maximally Stable Extremal Regions (MSER) and KAZE algorithms, were tested and parameters optimized. Image matching performance of these algorithms informed the decision to pursue further experiments with only SURF and KAZE. Optimally parametrized SURF and KAZE algorithms obtained co-registration accuracies of 0.1 and 0.3 pixels for intra-epoch and inter-epoch images respectively. To obtain better intra-epoch co-registration accuracy, collective band processing is advised whereas one-to-one matching strategy is recommended for inter-epoch co-registration. The results were tested using a maize crop monitoring case and the; comparison of spectral response of vegetation between the UAV sensors, Parrot Sequoia and Micro MCA was performed. Due to the missing incidence sensor, spectral and radiometric calibration of Micro MCA imagery is observed to be key in achieving optimal response. Also, the cameras have different specifications and thus differ in the quality of their respective photogrammetric outputs.

Deep learning-based models for building delineation from remotely sensed images face the challenge of producing precise and regular building outlines. This study investigates the combination of normalized digital surface models (nDSMs) with aerial images to optimize the extraction of building polygons using the frame field learning method. Results are evaluated at pixel, object, and polygon levels. In addition, an analysis is performed to assess the statistical deviations in the number of vertices of building polygons compared with the reference. The comparison of the number of vertices focuses on finding the output polygons that are the easiest to edit by human analysts in operational applications. It can serve as guidance to reduce the post-processing workload for obtaining high-accuracy building footprints. Experiments conducted in Enschede, the Netherlands, demonstrate that by introducing nDSM, the method could reduce the number of false positives and prevent missing the real buildings on the ground. The positional accuracy and shape similarity was improved, resulting in better-aligned building polygons. The method achieved a mean intersection over union (IoU) of 0.80 with the fused data (RGB + nDSM) against an IoU of 0.57 with the baseline (using RGB only) in the same area. A qualitative analysis of the results shows that the investigated model predicts more precise and regular polygons for large and complex structures.

The continuous urbanisation in most Low-to-Middle-Income-Country (LMIC) cities is accompanied by rapid socio-economic changes in urban and peri-urban areas. Urban transformation processes, such as gentrification as well as the increase in poor urban neighbourhoods (e.g., slums) produce new urban patterns. The intersection of very rapid socio-economic and demographic dynamics are often insufficiently understood, and relevant data for understanding them are commonly unavailable, dated, or too coarse (resolution). Traditional survey-based methods (e.g., census) are carried out at low temporal granularity and do not allow for frequent updates of large urban areas. Researchers and policymakers typically work with very dated data, which do not reflect on-the-ground realities and data aggregation hide socio-economic disparities. Therefore, the potential of Earth Observations (EO) needs to be unlocked. EO data have the ability to provide information at detailed spatial and temporal scales so as to support monitoring transformations. In this paper, we showcase how recent innovations in EO and Artificial Intelligence (AI) can provide relevant, rapid information about socio-economic conditions, and in particular on poor urban neighbourhoods, when large scale and/or multi-temporal data are required, e.g., to support Sustainable Development Goals (SDG) monitoring. We provide solutions to key challenges, including the provision of multi-scale data, the reduction in data costs, and the mapping of socio-economic conditions. These innovations fill data gaps for the production of statistical information, addressing the problems of access to field-based data under COVID-19.

Globally, about one billion urban dwellers live in deprived areas (commonly referred to as slums). However, this figure is highly uncertain due to large data gaps. For example, in many cities, systematic underreporting occurs, which hampers the monitoring of SDG indicators. Earth observation (EO) data can be used to extract consistent spatial information on important aspects of the physical domain of deprivation and can offer essential proxies to not well-covered (eg, social and economic) domains. However, for the development of a global data repository on deprived areas, several conceptual and methodological issues need to be solved. First, the relationship between concepts of a slum household and a deprived area needs to be defined in the context of information available in EO images. Second, the costs and benefits of different types of EO-data need to be established. Third, at different scales (ranging from communities, city to global scales), meaningful spatial aggregation units need to be established that are suitable to deal with uncertainties, privacy, ethics, and user needs. Fourth, computationally feasible, scalable, and transferable methods are required to produce a global data repository on deprived areas. This chapter provides an overview of methodological advances to address these four major challenges.

Deep learning has successfully improved the classification accuracy of optical remote sensing images. Recent works attempted to transfer the success of these techniques to the microwave domain to classify Polarimetric SAR data. So far, most deep learning networks separate amplitude and phase as separate input images. In this article, we present a deep fully convolutional network that uses real-valued weight kernels to perform pixel-wise classification of complex-valued images. We evaluated the performance of this network by comparing it with support vector machine, Random Forest, complex-valued convolutional neural network (CV-CNN), and a network that uses amplitude and phase information separately as real channels. The evaluation was done on a quad-polarized AIRSAR image and a dual-polarimetric multitemporal Sentinel-1 data acquired over Flevoland, the Netherlands. The proposed method achieved higher accuracy compared to all other networks with the same architecture.

Deep learning-based semantic segmentation models for building delineation face the challenge of producing precise and regular building outlines. Recently, a building delineation method based on frame field learning was proposed by Girard et al. (2020) to extract regular building footprints as vector polygons directly from aerial RGB images. A fully convolution network (FCN) is trained to learn simultaneously the building mask, contours, and frame field followed by a polygonization method. With the direction information of the building contours stored in the frame field, the polygonization algorithm produces regular outlines accurately detecting edges and corners. This paper investigated the contribution of elevation data from the normalized digital surface model (nDSM) to extract accurate and regular building polygons. The 3D information provided by the nDSM overcomes the aerial images’ limitations and contributes to distinguishing the buildings from the background more accurately. Experiments conducted in Enschede, the Netherlands, demonstrate that the nDSM improves building outlines’ accuracy, resulting in better-aligned building polygons and prevents false positives. The investigated deep learning approach (fusing RGB + nDSM) results in a mean intersection over union (IOU) of 0.70 in the urban area. The baseline method (using RGB only) results in an IOU of 0.58 in the same area. A qualitative analysis of the results shows that the investigated model predicts more precise and regular polygons for large and complex structures.

Deep learning methods based upon convolutional neural networks (CNNs) have demonstrated impressive performance in the task of building outline delineation from very high resolution (VHR) remote sensing (RS) imagery. In this paper, we introduce an improved method that is able to predict regularized building outline in a vector format within an end-to-end deep learning framework. The main idea of our framework is to learn to predict the location of key vertices of the buildings and connect them in sequence. The proposed method is based on PolyMapper. We upgrade the feature extraction by introducing global context and boundary refinement blocks and add channel and spatial attention modules to improve the effectiveness of the detection module. In addition, we introduce stacked conv-GRU to further preserve the geometric relationship between vertices and accelerate inference. We tested our method on two large-scale VHR-RS building extraction dataset. The results on both COCO and PoLiS metrics demonstrate better performance compared with Mask R-CNN and PolyMapper. Specifically, we achieve 4.2 mask mean average precision (mAP) and 3.7 mean average recall (mAR) absolute improvements compared to PolyMapper. Also, the qualitative comparison shows that our method significantly improves the instance segmentation of buildings of various shapes.

Achieving optimal yield and quality at harvest depends on the grower’s ability to avoid abiotic stresses (water, light, and temperature). This task has usually been satisfied through the implementation of adequate horticultural practices. In the context of clean energy transition and global climate change, growers nowadays have the possibility to grow their crops under solar panels, which modify the micro-environment of the crops. Being able to anticipate the behavior of plants under these new micro-environmental conditions would help growers adapt their horticultural practices. For electricity producers, in the context of dynamic agrivoltaic systems, anticipating the crop status is useful to choose a solar panels steering policy that maximizes electricity production while ensuring favorable environmental conditions for the crop to grow. To help electricity producers and growers estimate a crop status under panels, we developed a decision support system (DSS) called crop_sim. As of now, it can be used to monitor two types of perennial crops: grapevines and apple trees. crop_sim produces three indicators of the crop status: predawn water potential, canopy temperature and carbon production. Besides providing information on the crop status, the DSS incorporates an expert system which indicates the best time and the amount of irrigation to maintain a desired water status under the new micro-environmental conditions.This paper first focuses on the description of crop_sim and the usefulness of the three indicators. Then, a case study is presented. Our results show that, in a mature vineyard, with a typical panel steering policy conservative on crop yield, growers could save 13% of water compared to an open-field reference.Experimental data pertaining to apple trees, grapevines, tomatoes, and maize are being collected. They will be used to adapt the model to tomato and maize, evaluate it and make it robust enough to bring to market. Further improvements of the crop_sim model may be required to finely reproduce observations in the field. A full validation of the model is expected when all data from the experiments will be available. The DSS will evolve depending on the requirements of the agrivoltaics community and may incorporate additional plant indicators and new expert system rules.

Building extraction from remote sensing images using convolutional neural networks (CNNs) has been an active research topic in recent years. Most results obtained by CNN-based algorithms, however, still have common issues with the precision of the delineation of building outlines and the separation of different buildings. Recently, efforts have been made towards the automation of building outline regularization. This paper employs a new instance segmentation framework named Hybrid Task Cascade (HTC) as baseline model, integrating detection and segmentation as a joint multi-stage processing. We further integrate regularization methods such as convex hull and Douglas-Peucker algorithm to obtain accurately segmented edges. The method is tested on the crowdAI benchmark dataset by comparing with alternative state-of-the-art models (i.e., Mask R-CNN). The results show that our method achieves better instance segmentation results and improves the results in terms of geometric regularity of building segments. © 2020 IEEE.

Updated information on urban land use allows city planners and decision makers to conduct large scale monitoring of urban areas for sustainable urban growth. Remote sensing data and classification methods offer an efficient and reliable way to update such land use maps. Features extracted from land cover maps are helpful on performing a land use classification task. Such prior information can be embedded in the design of a deep learning based land use classifier by applying a multitask learning setup—simultaneously solving a land use and a land cover classification task. In this study, we explore a fully convolutional multitask network to classify urban land use from very high resolution (VHR) imagery. We experimented with three different setups of the fully convolutional network and compared it against a baseline random forest classifier. The first setup is a standard network only predicting the land use class of each pixel in the image. The second setup is a multitask network that concatenates the land use and land cover class labels in the same output layer of the network while the other setup accept as an input the land cover predictions, predicted by a subpart of the network, concatenated to the original input image patches. The two deep multitask networks outperforms the other two classifiers by at least 30% in average F1-score.

Existing algorithms for Digital Terrain Model (DTM) extraction still face difficulties due to data outliers and geometric ambiguities in the scene such as contiguous off-ground areas or sloped environments. We postulate that in such challenging cases, the radiometric information contained in aerial imagery may be leveraged to distinguish between ground and off-ground objects. We propose a method for DTM extraction from imagery which first applies morphological filters to the Digital Surface Model to obtain candidate ground and off-ground training samples. These samples are used to train a Fully Convolutional Network (FCN) in the second step, which can then be used to identify ground samples for the entire dataset. The proposed method harnesses the power of state-of-the-art deep learning methods, while showing how they can be adapted to the application of DTM extraction by (i) automatically selecting and labelling dataset-specific samples which can be used to train the network, and (ii) adapting the network architecture to consider a larger surface area without unnecessarily increasing the computational burden. The method is successfully tested on four datasets, indicating that the automatic labelling strategy can achieve an accuracy which is comparable to the use of manually labelled training samples. Furthermore, we demonstrate that the proposed method outperforms two reference DTM extraction algorithms in challenging areas.

Deep Learning has been massively used for image classification in recent years. The use of deep learning for ground classification from LIDAR point clouds has also been recently studied. However, point clouds need to be converted into an image in order to use Convolutional Neural Networks (CNNs). In state-of-the-art techniques, this conversion is slow because each point is converted into a separate image. This approach leads to highly redundant computation during conversion and classification. The goal of this study is to design a more efficient data conversion and ground classification. This goal is achieved by first converting the whole point cloud into a single image. The classification is then performed by a Fully Convolutional Network (FCN), a modified version of CNN designed for pixel-wise image classification. The proposed method is significantly faster than state-of-the-art techniques. On the ISPRS Filter Test dataset, it is 78 times faster for conversion and 16 times faster for classification. Our experimental analysis on the same dataset shows that the proposed method results in 5.22 % of total error, 4.10 % of type I error, and 15.07 % of type II error. Compared to the previous CNN-based technique and LAStools software, the proposed method reduces the total error and type I error (while type II error is slightly higher). The method was also tested on a very high point density LIDAR point clouds resulting in 4.02 % of total error, 2.15 % of type I error and 6.14 % of type II error.

Boundaries of agricultural fields are important features necessary for defining the location, shape, and spatial extent of agricultural units. They are commonly used to summarize production statistics at the field level. In this study, we investigate the delineation of agricultural field boundaries (AFB) from Sentinel-2 satellite images acquired over the Flevoland province, the Netherlands, using a deep learning technique based on fully convolutional networks (FCNs). We designed a multiple dilation fully convolutional network (MD-FCN) for AFB detection from Sentinel-2 images at 10 m resolution. Furthermore, we developed a novel super-resolution semantic contour detection network (named SRC-Net) using a transposed convolutional layer in the FCN architecture to enhance the spatial resolution of the AFB output from 10 m to 5 m resolution. The SRC-Net also improves the AFB maps at 5 m resolution by exploiting the spatial-contextual information in the label space. The results of the proposed SRC-Net outperform alternative upsampling techniques and are only slightly inferior to the results of the MD-FCN for AFB detection from RapidEye images acquired at 5 m resolution.

The cadastre is the foundation of land management. However, it is estimated that 70% of the land rights in the world remain unregistered. Traditional approaches are costly and labor intensive, therefore, recently the use of remotely sensed images has been investigated. The delineation of cadastral boundaries from such data is challenging since not all boundaries are demarcated by visible physical objects. In this paper, we introduce a technique based on deep Fully Convolutional Networks (FCNs), which can automatically learn high-level spatial features from images, to extract cadastral boundaries. Our strategy combines FCN and a grouping algorithm using the Oriented Watershed Transform (OWT) to generate connected contours. We carried out an experimental analysis in a real case study in Busogo, Rwanda, using images acquired by Unmanned Aerial Vehicles (UAV) in 2018. Our investigation shows promising results in automatically extracting visible boundaries, which can contribute to the current mapping and updating practices in Rwanda.

Detecting topographic changes in an urban environment and keeping city-level point clouds up-to-date are important tasks for urban planning and monitoring. In practice, remote sensing data are often available only in different modalities for two epochs. Change detection between airborne laser scanning data and photogrammetric data is challenging due to the multi-modality of the input data and dense matching errors. This paper proposes a method to detect building changes between multimodal acquisitions. The multimodal inputs are converted and fed into a light-weighted pseudo-Siamese convolutional neural network (PSI-CNN) for change detection. Different network configurations and fusion strategies are compared. Our experiments on a large urban data set demonstrate the effectiveness of the proposed method. Our change map achieves a recall rate of 86.17%, a precision rate of 68.16%, and an F1-score of 76.13%. The comparison between Siamese architecture and feed-forward architecture brings many interesting findings and suggestions to the design of networks for multimodal data processing.

Accurate spatial information of agricultural fields in smallholder farms is important for providing actionable information to farmers, managers, and policymakers. Very High Resolution (VHR) satellite images can capture such information. However, the automated delineation of fields in smallholder farms is a challenging task because of their small size, irregular shape and the use of mixed-cropping systems, which make their boundaries vaguely defined. Physical edges between smallholder fields are often indistinct in satellite imagery and contours need to be identified by considering the transition of the complex textural pattern between fields. In these circumstances, standard edge-detection algorithms fail to extract accurate boundaries. This article introduces a strategy to detect field boundaries using a fully convolutional network in combination with a globalisation and grouping algorithm. The convolutional network using an encoder-decoder structure is capable of learning complex spatial-contextual features from the image and accurately detects sparse field contours. A hierarchical segmentation is derived from the contours using the oriented watershed transform and by iteratively merging adjacent regions based on the average strength of their common boundary. Finally, field segments are obtained by adopting a combinatorial grouping algorithm exploiting the information of the segmentation hierarchy. An extensive experimental analysis is performed in two study areas in Nigeria and Mali using WorldView-2/3 images and comparing several state-of-the-art contour detection algorithms. The algorithms are compared based on the precision-recall accuracy assessment strategy which is tolerating small localisation errors in the detected contours. The proposed strategy shows promising results by automatically delineating field boundaries with F-scores higher than 0.7 and 0.6 on our two test areas, respectively, outperforming alternative techniques., Highlights • Deep learning technique to automatically delineate agricultural fields in smallholder farms • Extensive experimental analysis on two real case studies in Nigeria and Mali • Precision-recall framework for accuracy assessment introduced in remote sensing • The proposed technique compares favourably against state-of-the-art algorithms.

There is a growing demand for cheap and fast cadastral mapping methods to face the challenge of 70% global unregistered land rights. As traditional on-site field surveying is time-consuming and labor intensive, imagery-based cadastral mapping has in recent years been advocated by fit-for-purpose (FFP) land administration. However, owing to the semantic gap between the high-level cadastral boundary concept and low-level visual cues in the imagery, improving the accuracy of automatic boundary delineation remains a major challenge. In this research, we use imageries acquired by Unmanned Aerial Vehicles (UAV) to explore the potential of deep Fully Convolutional Networks (FCNs) for cadastral boundary detection in urban and semi-urban areas. We test the performance of FCNs against other state-of-the-art techniques, including Multi-Resolution Segmentation (MRS) and Globalized Probability of Boundary (gPb) in two case study sites in Rwanda. Experimental results show that FCNs outperformed MRS and gPb in both study areas and achieved an average accuracy of 0.79 in precision, 0.37 in recall and 0.50 in F-score. In conclusion, FCNs are able to effectively extract cadastral boundaries, especially when a large proportion of cadastral boundaries are visible. This automated method could minimize manual digitization and reduce field work, thus facilitating the current cadastral mapping and updating practices.

Automated delineation of smallholder farm fields is difficult because of their small size, irregular shape and the use of mixed-cropping systems. Edges between smallholder plots are often indistinct in satellite imagery and contours have to be identified by considering the transition of the complex textural patterns of the fields. We introduce a strategy to delineate field boundaries using a fully convolutional network in combination with a globalization and grouping algorithm to produce a hierarchical segmentation of the fields. We carry out an experimental analysis in a study area in Kofa, Nigeria, using a WorldView-3 image, comparing several state-of-the-art contour detection algorithms. The proposed strategy outperforms state-of-the-art computer vision methods and shows promising results by automatically delineating field boundaries with an accuracy close to human level photo-interpretation.

Cloud cover creates obstruction in Earth Observation studies. The obstruction is harder to distinguish from features having similar reflectance on the ground, such as snow. To distinguish clouds from snow in a VNIR image, we use an additional SWIR band. The images were fed into a deep Fully Convolutional Network that can fuse the multiresolution SWIR and VNIR bands together, in order to produce pixelwise classification. The accuracy obtained by the model on the test image was 93.35%. We compare the performance of this model with a more commonly used technique, Random Forests. To analyze the effect of SWIR, we use another deep learning model, trained only on the VNIR image, and compare the accuracies obtained.

Airborne photogrammetry and airborne laser scanning are two commonly used technologies used for topographical data acquisition at the city level. Change detection between airborne laser scanning data and photogrammetric data is challenging since the two point clouds show different characteristics. After comparing the two types of point clouds, this paper proposes a feed-forward Convolutional Neural Network (CNN) to detect building changes between them. The motivation from an application point of view is that the multimodal point clouds might be available for different epochs. Our method contains three steps: First, the point clouds and orthoimages are converted to raster images. Second, square patches are cropped from raster images and then fed into CNN for change detection. Finally, the original change map is post-processed with a simple connected component analysis. Experimental results show that the patch-based recall rate reaches 0.8146 and the precision rate reaches 0.7632. Object-based evaluation shows that 74 out of 86 building changes are correctly detected.

In very high resolution (VHR) remote sensing (RS) classification tasks, conventional pixel-based contextual information extraction methods such as morphological profiles (MPs), extended MPs (EMPs) and MPs with partial reconstruction (MPPR) with limited numbers, sizes and shapes of structural elements (SEs) cannot perfectly match all sizes and shapes of the objects in an image. To overcome such limitation, we introduce novel spatial feature extractors, namely, the superpixel-guided morphological profiles (SPMPs), where the superpixels are used as SEs in opening by reconstruction and closing by reconstruction operations. Moreover, to avoid possible side effects from unusual maximum and minimum values within superpixels, the mean pixel value of superpixels is adopted (SPMPsM). Additionally, new decision forest based on penalizing the attributes in previous trees, the ForestPA is introduced and evaluated through a comparative investigation on three VHR multi-/hyperspectral RS image classification tasks. Support vector machine and benchmark ensemble classifiers, including bagging, AdaBoost, MultiBoost, ExtraTrees, Random Forest and Rotation Forest, are adopted. The experimental results confirm the effectiveness and superior performances of the proposed SPMPs and SPMPsM relative to those of the MPs and MPPR. Moreover, ForestPA outperforms only bagging and is not suitable for learning from large numbers of samples with high dimensionality from the computational efficiency and classification accuracy perspective.

We are living in an urban age [...]

The success of supervised classification of remotely sensed images acquired over large geographical areas or at short time intervals strongly depends on the representativity of the samples used to train the classification algorithm and to define the model. When training samples are collected from an image (or a spatial region) different from the one used for mapping, spectral shifts between the two distributions are likely to make the model fail. Such shifts are generally due to differences in acquisition and atmospheric conditions or to changes in the nature of the object observed. In order to design classification methods that are robust to data-set shifts, recent remote sensing literature has considered solutions based on domain adaptation (DA) approaches. Inspired by machine learning literature, several DA methods have been proposed to solve specific problems in remote sensing data classification. This paper provides a critical review of the recent advances in DA for remote sensing and presents an overview of methods divided into four categories: i) invariant feature selection; ii) representation matching; iii) adaptation of classifiers and iv) selective sampling. We provide an overview of recent methodologies, as well as examples of application of the considered techniques to real remote sensing images characterized by very high spatial and spectral resolution. Finally, we propose guidelines to the selection of the method to use in real application scenarios.

In-situ slum upgrading projects include infrastructural improvements such as new roads, which are perceived to improve the quality of life for the residents and encourage structural improvements at a household level. Although these physical changes are easily visible in satellite imagery, it is more difficult to track incremental improvements undertaken by the residents – which are perhaps more closely linked to the socio-economic development of the households themselves. The improved detail provided by imagery obtained from Unmanned Aerial Vehicles (UAVs) has the potential to monitor these more subtle changes in a settlement. This paper provides a framework which takes advantage of high-resolution imagery and a detailed elevation model from UAVs to detect changes in informal settlements. The proposed framework leverages expert knowledge to provide training labels for deep learning and thus avoids the cost of manual labelling. The semantic classification is then used to interpret a change mask and identify: new buildings, the creation of open spaces, and incremental roof upgrading in an informal settlement. The methodology is demonstrated on UAV imagery of an informal settlement in Kigali, Rwanda, successfully identifying changes between 2015 and 2017 with an Overall Accuracy of 95 % and correctly interpreting changes with an Overall Accuracy of 91 %. Results reveal that almost half the buildings in the settlement show visible changes in the roofing material, and 61 % of these changed less than 1m². This demonstrates the incremental nature of housing improvements in the settlement.

Classification of crop types from multi-temporal SAR data is a complex task because of the need to extract spatial and temporal features from images affected by speckle. Previous methods applied speckle filtering and then classification in two separate processing steps. This paper introduces fully convolutional networks (FCN) for pixel-wise classification of crops from multi-temporal SAR data. It applies speckle filtering and classification in a single framework. Furthermore, it also uses dilated kernels to increase the capability to learn long distance spatial dependencies. The proposed FCN was compared with a patch-based convolutional neural network (CNN) and support vector machine (SVM) classifiers. The proposed method performed better when compared with the patch-based CNN and SVM.

Classification of very high-resolution (VHR) satellite images has three major challenges: 1) inherent low intraclass and high interclass spectral similarities; 2) mismatching resolution of available bands; and 3) the need to regularize noisy classification maps. Conventional methods have addressed these challenges by adopting separate stages of image fusion, feature extraction, and postclassification map regularization. These processing stages, however, are not jointly optimizing the classification task at hand. In this paper, we propose a single-stage framework embedding the processing stages in a recurrent multiresolution convolutional network trained in an end-to-end manner. The feedforward version of the network, called FuseNet , aims to match the resolution of the panchromatic and multispectral bands in a VHR image using convolutional layers with corresponding downsampling and upsampling operations. Contextual label information is incorporated into FuseNet by means of a recurrent version called ReuseNet . We compared FuseNet and ReuseNet against the use of separate processing steps for both image fusions, e.g., pansharpening and resampling through interpolation and map regularization such as conditional random fields . We carried out our experiments on a land-cover classification task using a Worldview-03 image of Quezon City, Philippines, and the International Society for Photogrammetry and Remote Sensing 2-D semantic labeling benchmark data set of Vaihingen, Germany. FuseNet and ReuseNet surpass the baseline approaches in both the quantitative and qualitative results.

Various classification methods have been developed to extract meaningful information from Airborne Laser Scanner (ALS) point clouds. However, the accuracy and the computational efficiency of the existing methods need to be improved, especially for the analysis of large datasets (e.g., at regional or national levels). In this paper, we present a novel deep learning approach to ground classification for Digital Terrain Model (DTM) extraction as well as for multi-class land-cover classification, delivering highly accurate classification results in a computationally efficient manner. Considering the top–down acquisition angle of ALS data, the point cloud is initially projected on the horizontal plane and converted into a multi-dimensional image. Then, classification techniques based on Fully Convolutional Networks (FCN) with dilated kernels are designed to perform pixel-wise image classification. Finally, labels are transferred from pixels to the original ALS points. We also designed a Multi-Scale FCN (MS-FCN) architecture to minimize the loss of information during the point-to-image conversion. In the ground classification experiment, we compared our method to a Convolutional Neural Network (CNN)-based method and LAStools software. We obtained a lower total error on both the International Society for Photogrammetry and Remote Sensing (ISPRS) filter test benchmark dataset and AHN-3 dataset in the Netherlands. In the multi-class classification experiment, our method resulted in higher precision and recall values compared to the traditional machine learning technique using Random Forest (RF); it accurately detected small buildings. The FCN achieved precision and recall values of 0.93 and 0.94 when RF obtained 0.91 and 0.92, respectively. Moreover, our strategy significantly improved the computational efficiency of state-of-the-art CNN-based methods, reducing the point-to-image conversion time from 47 h to 36 min in our experiments on the ISPRS filter test dataset. Misclassification errors remained in situations that were not included in the training dataset, such as large buildings and bridges, or contained noisy measurements.

The survey-based slum mapping (SBSM) program conducted by the Indonesian government to reach the national target of “cities without slums” by 2019 shows mapping inconsistencies due to several reasons, e.g., the dependency on the surveyor’s experiences and the complexity of the slum indicators set. By relying on such inconsistent maps, it will be difficult to monitor the national slum upgrading program’s progress. Remote sensing imagery combined with machine learning algorithms could support the reduction of these inconsistencies. This study evaluates the performance of two machine learning algorithms, i.e., support vector machine (SVM) and random forest (RF), for slum mapping in support of the slum mapping campaign in Bandung, Indonesia. Recognizing the complexity in differentiating slum and formal areas in Indonesia, the study used a combination of spectral, contextual, and morphological features. In addition, sequential feature selection (SFS) combined with the Hilbert–Schmidt independence criterion (HSIC) was used to select significant features for classifying slums. Overall, the highest accuracy (88.5%) was achieved by the SVM with SFS using contextual, morphological, and spectral features, which is higher than the estimated accuracy of the SBSM. To evaluate the potential of machine learning-based slum mapping (MLBSM) in support of slum upgrading programs, interviews were conducted with several local and national stakeholders. Results show that local acceptance for a remote sensing-based slum mapping approach varies among stakeholder groups. Therefore, a locally adapted framework is required to combine ground surveys with robust and consistent machine learning methods, for being able to deal with big data, and to allow the rapid extraction of consistent information on the dynamics of slums at a large scale.

Unmanned aerial vehicles (UAVs) have the potential to obtain high-resolution aerial imagery at frequent intervals, making them a valuable tool for urban planners who require up-to-date basemaps. Supervised classification methods can be exploited to translate the UAV data into such basemaps. However, these methods require labeled training samples, the collection of which may be complex and time consuming. Existing spatial datasets can be exploited to provide the training labels, but these often contain errors due to differences in the date or resolution of the dataset from which these outdated labels were obtained. In this paper, we propose an approach for updating basemaps using global and local contextual cues to automatically remove unreliable samples from the training set, and thereby, improve the classification accuracy. Using UAV datasets over Kigali, Rwanda, and Dar es Salaam, Tanzania, we demonstrate how the amount of mislabeled training samples can be reduced by 44.1% and 35.5%, respectively, leading to a classification accuracy of 92.1% in Kigali and 91.3% in Dar es Salaam. To achieve the same accuracy in Dar es Salaam, between 50000 and 60000 manually labeled image segments would be needed. This demonstrates that the proposed approach of using outdated spatial data to provide labels and iteratively removing unreliable samples is a viable method for obtaining high classification accuracies while reducing the costly step of acquiring labeled training samples.

Unmanned Aerial Vehicles (UAVs) are capable of providing very high resolution and up-to-date information to support informal settlement upgrading projects. In order to provide accurate basemaps, urban scene understanding through the identification and classification of buildings and terrain is imperative. However, common characteristics of informal settlements such as small, irregular buildings with heterogeneous roof material and large presence of clutter challenge state-of-the-art algorithms. Especially the dense buildings and steeply sloped terrain cause difficulties in identifying elevated objects. This work investigates how 2D radiometric and textural features, 2.5D topographic features, and 3D geometric features obtained from UAV imagery can be integrated to obtain a high classification accuracy in challenging classification problems for the analysis of informal settlements. It compares the utility of pixel-based and segment-based features obtained from an orthomosaic and DSM with point-based and segment-based features extracted from the point cloud to classify an unplanned settlement in Kigali, Rwanda. Findings show that the integration of 2D and 3D features leads to higher classification accuracies.

Classification of Synthetic Aperture Radar (SAR) images is a complex task because of the presence of speckle, which affects images in a way similar to a strong noise. In this study, we investigate the use of Convolutional Neural Networks (CNNs) which can effectively learn a bank of spatial filters to simultaneously 1) reduce speckle noise, and 2) extract spatial-contextual features to characterize texture and scattering mechanism. Moreover, we combine CNN with Markov Random Fields (MRFs) for post-classification label smoothing to further reduce the effect of speckle on the land-cover map and to improve classification accuracy. We applied the proposed classification system to the analysis of a multitemporal series of Sentinel-1 images for mapping agricultural fields in Flevoland, The Netherlands. Experimental results confirm the effectiveness of the investigated approach, which outperforms standard methods.

Convolutional neural networks (CNNs), widely studied in the domain of computer vision, are more recently finding application in the analysis of high-resolution aerial and satellite imagery. In this paper, we investigate a deep feature learning approach based on CNNs for the detection of informal settlements in Dar es Salaam, Tanzania. This information is vital for decision making and planning of upgrading processes. Distinguishing the different urban structure types is challenging because of the abstract semantic definition of the classes as opposed to the separation of standard land-cover classes. This task requires the extraction of complex spatial-contextual features. To this aim, we trained a CNN in an end-to-end fashion and used it to classify informal and formal settlements. Our experimental results show that CNNs outperform state of the art methods using hand-crafted features. We conclude that CNNs are able to effectively learn the spatial-contextual features for accurately discriminating formal and informal settlements.

The increased reliance on geospatial data for decision-making in urban planning makes it imperative that the available spatial information is up-to-date and faithfully represents reality. This calls for map updating methods which support the integration of data from different sources in an automated manner. In this paper, we utilize existing basemap information to provide the initial data labels, thus reducing the lengthy process of label acquisition. However, we take into account that a portion of these labels are likely to be incorrect due to changes such as new constructions. We then cast the updating problem as a supervised classification with noisy training labels. Through an iterative approach, training samples which rank low on two criteria (label consistency and contextual consistency) are considered to be unreliable and removed from the training set. This technique is demonstrated in the specific context in which data obtained from an Unmanned Aerial Vehicle (UAV) is used to update building outlines in an informal settlement in Kigali, Rwanda. The proposed approach is able to accurately classify 95.34% of the UAV imagery even though the original labels are based on data obtained from outdated aerial imagery of a lower spatial resolution, causing 14.3% of the segments to have an incorrect training label. In this paper, we describe the proposed method, demonstrate the importance of both the contextual consistency and label consistency for filtering the training samples, discuss the robustness of the method to noise levels, and discuss the implications of this approach for other applications.

This letter investigates fully convolutional networks (FCNs) for the detection of informal settlements in very high resolution (VHR) satellite images. Informal settlements or slums are proliferating in developing countries and their detection and classification provides vital information for decision making and planning urban upgrading processes. Distinguishing different urban structures in VHR images is challenging because of the abstract semantic definition of the classes as opposed to the separation of standard land-cover classes. This task requires extraction of texture and spatial features. To this aim, we introduce deep FCNs to perform pixel-wise image labeling by automatically learning a higher level representation of the data. Deep FCNs can learn a hierarchy of features associated to increasing levels of abstraction, from raw pixel values to edges and corners up to complex spatial patterns. We present a deep FCN using dilated convolutions of increasing spatial support. It is capable of learning informative features capturing long-range pixel dependencies while keeping a limited number of network parameters. Experiments carried out on a Quickbird image acquired over the city of Dar es Salaam, Tanzania, show that the proposed FCN outperforms state-of-the-art convolutional networks. Moreover, the computational cost of the proposed technique is significantly lower than standard patch-based architectures.

Information about the location and extent of informal settlements is necessary to guide decision making and resource allocation for their upgrading. Very high resolution (VHR) satellite images can provide this useful information, however, different urban settlement types are hard to be automatically discriminated and extracted from VHR imagery, because of their abstract semantic class definition. State-of-the-art classification techniques rely on hand-engineering spatial-contextual features to improve the classification results of pixel-based methods. In this paper, we propose to use convolutional neural networks (CNNs) for learning discriminative spatial features, and perform automatic detection of informal settlements. The experimental analysis is carried out on a QuickBird image acquired over Dar es Salaam, Tanzania. The proposed technique is compared against support vector machines (SVMs) using texture features extracted from grey level co-occurrence matrix (GLCM) and local binary patterns (LBP), which result in accuracies of 86.65% and 90.48%, respectively. CNN leads to better classification, resulting in an overall accuracy of 91.71%. A sensitivity analysis shows that deeper networks result in higher accuracies when large training sets are used. The study concludes that training CNN in an end-to-end fashion can automatically learn spatial features from the data that are capable of discriminating complex urban land use classes.

This paper aims at analyzing and comparing active learning (AL) and semisupervised learning (SSL) methods for the classification of remote sensing (RS) images. We present a literature review of the two learning paradigms and compare them theoretically and experimentally when addressing classification problems characterized by few training samples (w.r.t. the number of features) and affected by sample selection bias. Commonalities and differences are highlighted in the context of a conceptual framework used to describe the workflow of the two approaches. We point out advantages and disadvantages of the two approaches, delineating the boundary conditions on the applicability of the two paradigms with respect to both the amount and the quality of available training samples. Moreover, we investigate the integration of concepts that are in common between the two learning paradigms for improving state-of-the-art techniques and combining AL and SSL in order to jointly leverage the advantages of both approaches. In this framework, we propose a novel SSL algorithm that improves the progressive semisupervised support vector machine by integrating concepts that are usually considered in AL methods. We performed several experiments considering both synthetic and real multispectral and hyperspectral RS data, defining different classification problems starting from different initial training sets. The experiments are carried out considering classification methods based on support vector machines.

In the cities of the Global South, slum settlements are growing in size and number, but their locations and characteristics are often missing in official statistics and maps. Although several studies have focused on detecting slums from satellite images, only a few captured their variations. This study addresses this gap using an integrated approach that can identify a slums’ degree of deprivation in terms of socio-economic variability in Bangalore, India using image features derived from very high resolution (VHR) satellite images. To characterize deprivation, we use multiple correspondence analysis (MCA) and quantify deprivation with a data-driven index of multiple deprivation (DIMD). We take advantage of spatial features learned by a convolutional neural network (CNN) from VHR satellite images to predict the DIMD. To deal with a small training dataset of only 121 samples with known DIMD values, insufficient to train a deep CNN, we conduct a two-step transfer learning approach using 1461 delineated slum boundaries as follows. First, a CNN is trained using these samples to classify slums and formal areas. The trained network is then fine-tuned using the 121 samples to directly predict the DIMD. The best prediction is obtained by using an ensemble non-linear regression model, combining the results of the CNN and models based on hand-crafted and geographic information system (GIS) features, with R2 of 0.75. Our findings show that using the proposed two-step transfer learning approach, a deep CNN can be trained with a limited number of samples to predict the slums’ degree of deprivation. This demonstrates that the CNN-based approach can capture variations of deprivation in VHR images, providing a comprehensive understanding of the socio-economic situation of slums in Bangalore.

Spatial-contextual features play a vital role in the classification of very high resolution aerial images characterized by sub-decimetre resolution. However, manually extracting relevant contextual features is difficult and time-consuming in the analysis of sub-decimetre resolution images, where the objects of interest are significantly larger than the pixel size. Deep learning methods allow us to replace hand-crafted features by automatically learning contextual features from the image. In this paper, we investigate the use of convolutional neural networks (CNN) for the classification of urban areas using high resolution airborne images. We also analyse the sensitivity of network hyperparameters providing an interpretation of their effect on the extraction of spatial-contextual features. Experimental results show the effectiveness of CNN in learning discriminative contextual features leading to accurate classified maps and outperforming traditional classification methods based on the extraction of textural features.

This paper presents a two-level Active Learning (AL) classification method for the interactive detection of earthquake-induced debris via the synergetic use of post-disaster Very High Resolution (VHR) satellite and local decimeter-resolution aerial images. The proposed method is performed by interactively guiding the human expert in the collection of labeled training samples from aerial images and optimally planning further aerial surveys. A label propagation mechanism is adopted to reliably propagate the labeled pixels annotated by the user on the aerial image by visual photointerpretation to the (lower resolution) satellite image. The experimental analysis is carried out using post-disaster images of the 2010 Yushu earthquake in China. The obtained results confirm that the proposed method can significantly reduce the user annotation effort and the cost for acquiring additional aerial images leading to more accurate and timely debris detection maps.

Unmanned Aerial Vehicles (UAVs) are capable of providing high-quality orthoimagery and 3D information in the form of point clouds at a relatively low cost. Their increasing popularity stresses the necessity of understanding which algorithms are especially suited for processing the data obtained from UAVs. The features that are extracted from the point cloud and imagery have different statistical characteristics and can be considered as heterogeneous, which motivates the use of Multiple Kernel Learning (MKL) for classification problems. In this paper, we illustrate the utility of applying MKL for the classification of heterogeneous features obtained from UAV data through a case study of an informal settlement in Kigali, Rwanda. Results indicate that MKL can achieve a classification accuracy of 90.6%, a 5.2% increase over a standard single-kernel Support Vector Machine (SVM). A comparison of seven MKL methods indicates that linearly-weighted kernel combinations based on simple heuristics are competitive with respect to computationally-complex, non-linear kernel combination methods. We further underline the importance of utilizing appropriate feature grouping strategies for MKL, which has not been directly addressed in the literature, and we propose a novel, automated feature grouping method that achieves a high classification accuracy for various MKL methods.

This paper presents a novel protocol for the accuracy assessment of the thematic maps obtained by the classification of very high resolution images. As the thematic accuracy alone is not sufficient to adequately characterize the geometrical properties of high-resolution classification maps, we propose a protocol that is based on the analysis of two families of indices: 1) the traditional thematic accuracy indices and 2) a set of novel geometric indices that model different geometric properties of the objects recognized in the map. In this context, we present a set of indices that characterize five different types of geometric errors in the classification map: 1) oversegmentation; 2) undersegmentation; 3) edge location; 4) shape distortion; and 5) fragmentation. Moreover, we propose a new approach for tuning the free parameters of supervised classifiers on the basis of a multiobjective criterion function that aims at selecting the parameter values that result in the classification map that jointly optimize thematic and geometric error indices. Experimental results obtained on QuickBird images show the effectiveness of the proposed protocol in selecting classification maps characterized by a better tradeoff between thematic and geometric accuracies than standard procedures based only on thematic accuracy measures. In addition, results obtained with support vector machine classifiers confirm the effectiveness of the proposed multiobjective technique for the selection of free-parameter values for the classification algorithm.

Active learning typically aims at minimizing the number of labeled samples to be included in the training set to reach a certain level of classification accuracy. Standard methods do not usually take into account the real annotation procedures and implicitly assume that all samples require the same effort to be labeled. Here, we consider the case where the cost associated with the annotation of a given sample depends on the previously labeled samples. In general, this is the case when annotating a queried sample is an action that changes the state of a dynamic system, and the cost is a function of the state of the system. In order to minimize the total annotation cost, the active sample selection problem is addressed in the framework of a Markov decision process, which allows one to plan the next labeling action on the basis of an expected long-term cumulative reward. This framework allows us to address the problem of optimizing the collection of labeled samples by field surveys for the classification of remote sensing data. The proposed method is applied to the ground sample collection for tree species classification using airborne hyperspectral images. Experiments carried out in the context of a real case study on forest inventory show the effectiveness of the proposed method.

This study presents a cost-sensitive active learning method for optimizing the field surveys by a human expert in the classification of single tree species using hyperspectral images. The goal of the proposed method is to guide the human expert in the collection of labeled samples in order to maximize the ratio between the classification accuracy with respect to the travelling costs. Experiments carried out in the context of a real study on forest inventory show the effectiveness of the proposed method.

This paper presents a novel interactive domain-adaptation technique based on active learning for the classification of remote sensing (RS) images. The proposed method aims at adapting the supervised classifier trained on a given RS source image to make it suitable for classifying a different but related target image. The two images can be acquired in different locations and/or at different times, but present the same set of land-cover classes. The proposed approach iteratively selects the most informative samples of the target image to be labeled by the user and included in the training set, while the source-image samples are re-weighted or possibly removed from the training set on the basis of their disagreement with the target image classification problem. In this way, the consistent information available from the source image can be effectively exploited for the classification of a target image and for guiding the user in the selection of the new samples to be labeled, whereas the inconsistent information is automatically detected and removed. Experimental results on a Very High Resolution (VHR) multispectral dataset confirm the effectiveness of the proposed method.

This paper presents a comparative study in order to analyze active learning (AL) and semi-supervised learning (SSL) for the classification of remote sensing (RS) images. The two learning paradigms are analyzed both from the theoretical and experimental point of view. The aim of this work is to identify the advantages and disadvantages of AL and SSL methods, and to point out the boundary conditions on the applicability of these methods with respect to both the number of available labeled samples and the reliability of classification results. In our experimental analysis, AL and SSL techniques have been applied to the classification of both synthetic and real RS data, defining different classification problems starting from different initial training sets and considering different distributions of the classes. This analysis allowed us to derive important conclusion about the use of these classification approaches and to obtain insight about which one of the two approaches is more appropriate according to the specific classification problem, the available initial training set and the available budget for the acquisition of new labeled samples.

We present a novel technique for addressing domain adaptation problems in the classification of remote sensing images with active learning. Domain adaptation is the important problem of adapting a supervised classifier trained on a given image (source domain) to the classification of another similar (but not identical) image (target domain) acquired on a different area, or on the same area at a different time. The main idea of the proposed approach is to iteratively labeling and adding to the training set the minimum number of the most informative samples from target domain, while removing the source-domain samples that does not fit with the distributions of the classes in the target domain. In this way, the classification system exploits already available information, i.e., the labeled samples of source domain, in order to minimize the number of target domain samples to be labeled, thus reducing the cost associated to the definition of the training set for the classification of the target domain. Experimental results obtained in the classification of a hyperspectral image confirm the effectiveness of the proposed technique.