Search

Accurate urban land cover information is crucial for effective urban planning and management. While convolutional neural networks (CNNs) demonstrate superior feature learning and prediction capabilities using image-level annotations, the inherent mixed-category nature of input image patches leads to classification errors along object boundaries. Fully convolutional neural networks (FCNs) excel at pixel-wise fine segmentation, making them less susceptible to heterogeneous content, but they require fully annotated dense image patches, which may not be readily available in real-world scenarios. This paper proposes an object-based semi-supervised spatial attention residual UNet (OS-ARU) model. First, multiscale segmentation is performed to obtain segments from a remote sensing image, and segments containing sample points are assigned the categories of the corresponding points, which are used to train the model. Then, the trained model predicts class probabilities for all segments. Each unlabeled segment’s probability distribution is compared against those of labeled segments for similarity matching under a threshold constraint. Through label propagation, pseudo-labels are assigned to unlabeled segments exhibiting high similarity to labeled ones. Finally, the model is retrained using the augmented training set incorporating the pseudo-labeled segments. Comprehensive experiments on aerial image benchmarks for Vaihingen and Potsdam demonstrate that the proposed OS-ARU achieves higher classification accuracy than state-of-the-art models, including OCNN, 2OCNN, and standard OS-U, reaching an overall accuracy (OA) of 87.83% and 86.71%, respectively. The performance improvements over the baseline methods are statistically significant according to the Wilcoxon Signed-Rank Test. Despite using significantly fewer sparse annotations, this semi-supervised approach still achieves comparable accuracy to the same model under full supervision. The proposed method thus makes a step forw

The capsule network (Caps) is a novel type of neural network that has great potential for the classification of hyperspectral remote sensing. However, the Caps suffers from the issue of gradient vanishing. To solve this problem, a powered activation regularization based adaptive capsule network (PAR-ACaps) was proposed for hyperspectral remote sensing classification, in which an adaptive routing algorithm without iteration was applied to amplify the gradient, and the powered activation regularization method was used to learn the sparser and more discriminative representation. The classification performance of PAR-ACaps was evaluated using two public hyperspectral remote sensing datasets, i.e., the Pavia University (PU) and Salinas (SA) datasets. The average overall classification accuracy (OA) of PAR-ACaps with shallower architecture was measured and compared with those of the benchmarks, including random forest (RF), support vector machine (SVM), 1-dimensional convolutional neural network (1DCNN), two-dimensional convolutional neural network (CNN), three-dimensional convolutional neural network (3DCNN), Caps, and the original adaptive capsule network (ACaps) with comparable network architectures. The OA of PAR-ACaps for PU and SA datasets was 99.51% and 94.52%, respectively, which was higher than those of benchmarks. Moreover, the classification performance of PAR-ACaps with relatively deeper neural architecture (four and six convolutional layers in the feature extraction stage) was also evaluated to demonstrate the effectiveness of gradient amplification. As shown in the experimental results, the classification performance of PAR-ACaps with relatively deeper neural architecture for PU and SA datasets was also superior to 1DCNN, CNN, 3DCNN, Caps, and ACaps with comparable neural architectures. Additionally, the training time consumed by PAR-ACaps was significantly lower than that of Caps. The proposed PAR-ACaps is, therefore, recommended as an effective alternative

The highly dynamic nature of agro-ecosystems in space and time usually leads to high intra-class variance and low inter-class separability in the fine spatial resolution (FSR) remotely sensed imagery. This makes traditional classifiers essentially relying on spectral information for crop mapping from FSR imagery an extremely challenging task. To mine effectively the rich spectral and spatial information in FSR imagery, this paper proposed a Scale Sequence Object-based Convolutional Neural Network (SS-OCNN) that classifies images at the object level by taking segmented objects (crop parcels) as basic units of analysis, thus, ensuring that the boundaries between crop parcels are delineated precisely. These segmented objects were subsequently classified using a CNN model integrated with an automatically generated scale sequence of input patch sizes. This scale sequence can fuse effectively the features learned at different scales by transforming progressively the information extracted at small scales to larger scales. The effectiveness of the SS-OCNN was investigated using two heterogeneous agricultural areas with FSR SAR and optical imagery, respectively. Experimental results revealed that the SS-OCNN consistently achieved the most accurate classification results. The SS-OCNN, thus, provides a new paradigm for crop classification over heterogeneous areas using FSR imagery, and has a wide application prospect.

The majority of existing research on urban land cover and land use (LULC) classification from remotely sensed imagery is to differentiate at a specific level (either land cover (LC) or land use (LU)) and at a specific scale. The spatial and hierarchical relationships between LC and LU were not considered in the previous paradigm. This paper proposed Joint Deep Learning as a novel framework for urban LULC classification. Three state-of-the-art methods were compared under the framework, including JDL, SS-JDL and a newly proposed Cross GAN. Experimental results demonstrate the superiority of Cross GAN in terms of classification accuracy and robustness.

Choosing appropriate scales for remotely sensed image classification is extremely important yet still an open question in relation to deep convolutional neural networks (CNN), due to the impact of spatial scale (i.e., input patch size) on the recognition of ground objects. Currently, the optimal scale selection processes are extremely cumbersome and time-consuming requiring repetitive experiments involving trial-and-error procedures, which significantly reduces the practical utility of the corresponding classification methods. This issue is crucial when trying to classify large-scale land use (LU) and land cover (LC) jointly (Zhang et al., 2019). In this paper, a simple and parsimonious scale sequence joint deep learning (SS-JDL) method is proposed for joint LU and LC classification, in which a sequence of scales is embedded in the iterative process of fitting the joint distribution implicit in the joint deep learning (JDL) method, thus, replacing the previous paradigm of scale selection. The sequence of scales, derived autonomously and used to define the CNN input patch sizes, provides consecutive information transmission from small-scale features to large-scale representations, and from simple LC states to complex LU characterisations. The effectiveness of the novel SS-JDL method was tested on aerial digital photography of three complex and heterogeneous landscapes, two in Southern England (Bournemouth and Southampton) and one in North West England (Manchester). Benchmark comparisons were provided in the form of a range of LU and LC methods, including the state-of-the-art joint deep learning (JDL) method. The experimental results demonstrated that the SS-JDL consistently outperformed all of the state-of-the-art baselines in terms of both LU and LC classification accuracies, as well as computational efficiency. The proposed SS-JDL method, therefore, represents a fast and effective implementation of the state-of-the-art JDL method. By creating a single, unifying joi

Accurate information on crop distribution is of great importance for a range of applications including crop yield estimation, greenhouse gas emission measurement and management policy formulation. Fine spatial resolution (FSR) remotely sensed imagery provides new opportunities for crop mapping at a detailed level. However, crop classification from FSR imagery is known to be challenging due to the great intra-class variability and low inter-class disparity in the data. In this research, a novel hybrid method (OSVM-OCNN) was proposed for crop classification from FSR imagery, which combines a shallow-structured object-based support vector machine (OSVM) with a deep-structured object-based convolutional neural network (OCNN). Unlike pixel-wise classification methods, the OSVM-OCNN method operates on objects as the basic units of analysis and, thus, classifies remotely sensed images at the object level. The proposed OSVM-OCNN harvests the complementary characteristics of the two sub-models, the OSVM with effective extraction of low-level within-object features and the OCNN with capture and utilization of high-level between-object information. By using a rule-based fusion strategy based primarily on the OCNN’s prediction probability, the two sub-models were fused in a concise and effective manner. We investigated the effectiveness of the proposed method over two test sites (i.e., S1 and S2) that have distinctive and heterogeneous patterns of different crops in the Sacramento Valley, California, using FSR Synthetic Aperture Radar (SAR) and FSR multispectral data, respectively. Experimental results illustrated that the new proposed OSVM-OCNN approach increased markedly the classification accuracy for most of crop types in S1 and all crop types in S2, and it consistently achieved the most accurate accuracy in comparison with its two object-based sub-models (OSVM and OCNN) as well as the pixel-wise SVM (PSVM) and CNN (PCNN) methods. Our findings, thus, suggest that the propos

Land cover (LC) and land use (LU) have commonly been classified separately from remotely sensed imagery, without considering the intrinsically hierarchical and nested relationships between them. In this paper, for the first time, a highly novel joint deep learning framework is proposed and demonstrated for LC and LU classification. The proposed Joint Deep Learning (JDL) model incorporates a multilayer perceptron (MLP) and convolutional neural network (CNN), and is implemented via a Markov process involving iterative updating. In the JDL, LU classification conducted by the CNN is made conditional upon the LC probabilities predicted by the MLP. In turn, those LU probabilities together with the original imagery are re-used as inputs to the MLP to strengthen the spatial and spectral feature representations. This process of updating the MLP and CNN forms a joint distribution, where both LC and LU are classified simultaneously through iteration. The proposed JDL method provides a general framework within which the pixel-based MLP and the patch-based CNN provide mutually complementary information to each other, such that both are refined in the classification process through iteration. Given the well-known complexities associated with the classification of very fine spatial resolution (VFSR) imagery, the effectiveness of the proposed JDL was tested on aerial photography of two large urban and suburban areas in Great Britain (Southampton and Manchester). The JDL consistently demonstrated greatly increased accuracies with increasing iteration, not only for the LU classification, but for both the LC and LU classifications, achieving by far the greatest accuracies for each at around 10 iterations. The average overall classification accuracies were 90.18% for LC and 87.92% for LU for the two study sites, far higher than the initial accuracies and consistently outperforming benchmark comparators (three each for LC and LU classification). This research, thus, represents the first

Spatial and temporal information on plant and soil conditions is needed urgently for monitoring of crop productivity. Remote sensing has been considered as an effective means for crop growth monitoring due to its timely updating and complete coverage. In this paper, we explored the potential of L-band fully-polarimetric Uninhabited Aerial Vehicle Synthetic Aperture Radar (UAVSAR) data for crop monitoring and classification. The study site was located in the Sacramento Valley, in California where the cropping system is relatively diverse. Full season polarimetric signatures, as well as scattering mechanisms, for several crops, including almond, walnut, alfalfa, winter wheat, corn, sunflower, and tomato, were analyzed with linear polarizations (HH, HV, and VV) and polarimetric decomposition (Cloude–Pottier and Freeman–Durden) parameters, respectively. The separability amongst crop types was assessed across a full calendar year based on both linear polarizations and decomposition parameters. The unique structure-related polarimetric signature of each crop was provided by multitemporal UAVSAR data with a fine temporal resolution. Permanent tree crops (almond and walnut) and alfalfa demonstrated stable radar backscattering values across the growing season, whereas winter wheat and summer crops (corn, sunflower, and tomato) presented drastically different patterns, with rapid increase from the emergence stage to the peak biomass stage, followed by a significant decrease during the senescence stage. In general, the polarimetric signature was heterogeneous during June and October, while homogeneous during March-to-May and July-to-August. The scattering mechanisms depend heavily upon crop type and phenological stage. The primary scattering mechanism for tree crops was volume scattering (>40%), while surface scattering (>40%) dominated for alfalfa and winter wheat, although double-bounce scattering (>30%) was notable for alfalfa during March-to-September. Surface scattering w

Spatial and temporal information on plant and soil conditions is needed urgently for monitoring of crop productivity. Remote sensing has been considered as an effective means for crop growth monitoring due to its timely updating and complete coverage. In this paper, we explored the potential of L-band fully-polarimetric Uninhabited Aerial Vehicle Synthetic Aperture Radar (UAVSAR) data for crop monitoring and classification. The study site was located in the Sacramento Valley, in California where the cropping system is relatively diverse. Full season polarimetric signatures, as well as scattering mechanisms, for several crops, including almond, walnut, alfalfa, winter wheat, corn, sunflower, and tomato, were analyzed with linear polarizations (HH, HV, and VV) and polarimetric decomposition (Cloude–Pottier and Freeman–Durden) parameters, respectively. The separability amongst crop types was assessed across a full calendar year based on both linear polarizations and decomposition parameters. The unique structure-related polarimetric signature of each crop was provided by multitemporal UAVSAR data with a fine temporal resolution. Permanent tree crops (almond and walnut) and alfalfa demonstrated stable radar backscattering values across the growing season, whereas winter wheat and summer crops (corn, sunflower, and tomato) presented drastically different patterns, with rapid increase from the emergence stage to the peak biomass stage, followed by a significant decrease during the senescence stage. In general, the polarimetric signature was heterogeneous during June and October, while homogeneous during March-to-May and July-to-August. The scattering mechanisms depend heavily upon crop type and phenological stage. The primary scattering mechanism for tree crops was volume scattering (>40%), while surface scattering (>40%) dominated for alfalfa and winter wheat, although double-bounce scattering (>30%) was notable for alfalfa during March-to-September. Surface scattering w

Urban land use information is essential for a variety of urban-related applications such as urban planning and regional administration. The extraction of urban land use from very fine spatial resolution (VFSR) remotely sensed imagery has, therefore, drawn much attention in the remote sensing community. Nevertheless, classifying urban land use from VFSR images remains a challenging task, due to the extreme difficulties in differentiating complex spatial patterns to derive high-level semantic labels. Deep convolutional neural networks (CNNs) offer great potential to extract high-level spatial features, thanks to its hierarchical nature with multiple levels of abstraction. However, blurred object boundaries and geometric distortion, as well as huge computational redundancy, severely restrict the potential application of CNN for the classification of urban land use. In this paper, a novel object-based convolutional neural network (OCNN) is proposed for urban land use classification using VFSR images. Rather than pixel-wise convolutional processes, the OCNN relies on segmented objects as its functional units, and CNN networks are used to analyse and label objects such as to partition within-object and between-object variation. Two CNN networks with different model structures and window sizes are developed to predict linearly shaped objects (e.g. Highway, Canal) and general (other non-linearly shaped) objects. Then a rule-based decision fusion is performed to integrate the class-specific classification results. The effectiveness of the proposed OCNN method was tested on aerial photography of two large urban scenes in Southampton and Manchester in Great Britain. The OCNN combined with large and small window sizes achieved excellent classification accuracy and computational efficiency, consistently outperforming its sub-modules, as well as other benchmark comparators, including the pixel-wise CNN, contextual-based MRF and object-based OBIA-SVM methods. The proposed method p

The contextual-based convolutional neural network (CNN) with deep architecture and pixel-based multilayer perceptron (MLP) with shallow structure are well-recognized neural network algorithms, representing the state-of-the-art deep learning method and the classical non-parametric machine learning approach, respectively. The two algorithms, which have very different behaviours, were integrated in a concise and effective way using a rule-based decision fusion approach for the classification of very fine spatial resolution (VFSR) remotely sensed imagery. The decision fusion rules, designed primarily based on the classification confidence of the CNN, reflect the generally complementary patterns of the individual classifiers. In consequence, the proposed ensemble classifier MLP-CNN harvests the complementary results acquired from the CNN based on deep spatial feature representation and from the MLP based on spectral discrimination. Meanwhile, limitations of the CNN due to the adoption of convolutional filters such as the uncertainty in object boundary partition and loss of useful fine spatial resolution detail were compensated. The effectiveness of the ensemble MLP-CNN classifier was tested in both urban and rural areas using aerial photography together with an additional satellite sensor dataset. The MLP-CNN classifier achieved promising performance, consistently outperforming the pixel-based MLP, spectral and textural-based MLP, and the contextual-based CNN in terms of classification accuracy. This research paves the way to effectively address the complicated problem of VFSR image classification.

The rapid development of earth observation technology has produced large quantities of remote-sensing data. Unsupervised classification (i.e. clustering) of remote-sensing images, an important means to acquire land-use/cover information, has become increasingly in demand due to its simplicity and ease of application. Traditional methods, such as k-means, struggle to solve this NP-hard (Non-deterministic Polynomial hard) image classification problem. Particle swarm optimization (PSO), always achieving better result than k-means, has recently been applied to unsupervised image classification. However, PSO was also found to be easily trapped on local optima. This article proposes a novel unsupervised Levy flight particle swarm optimization (ULPSO) method for image classification with balanced exploitation and exploration capabilities. It benefits from a new searching strategy: the worst particle in the swarm is targeted and its position is updated with Levy flight at each iteration. The effectiveness of the proposed method was tested with three types of remote-sensing imagery (Landsat Thematic Mapper (TM), Flightline C1 (FLC), and QuickBird) that are distinct in terms of spatial and spectral resolution and landscape. Our results showed that ULPSO is able to achieve significantly better and more stable classification results than k-means and the other two intelligent methods based on genetic algorithm (GA) and particle swarm optimization (PSO) over all of the experiments. ULPSO is, therefore, recommended as an effective alternative for unsupervised remote-sensing image classification.

Recent advances in remote sensing have witnessed a great amount of very high resolution (VHR) images acquired at sub-metre spatial resolution. These VHR remotely sensed data has post enormous challenges in processing, analysing and classifying them effectively due to the high spatial complexity and heterogeneity. Although many computer-aid classification methods that based on machine learning approaches have been developed over the past decades, most of them are developed toward pixel level spectral differentiation, e.g. Multi-Layer Perceptron (MLP), which are unable to exploit abundant spatial details within VHR images. This paper introduced a rough set model as a general framework to objectively characterize the uncertainty in CNN classification results, and further partition them into correctness and incorrectness on the map. The correct classification regions of CNN were trusted and maintained, whereas the misclassification areas were reclassified using a decision tree with both CNN and MLP. The effectiveness of the proposed rough set decision tree based MLP-CNN was tested using an urban area at Bournemouth, United Kingdom. The MLP-CNN, well capturing the complementarity between CNN and MLP through the rough set based decision tree, achieved the best classification performance both visually and numerically. Therefore, this research paves the way to achieve fully automatic and effective VHR image classification.

The modeling and predicting of vegetation Leaf area index (LAI) is an important component of land surface model and assimilation of remote sensing data. The MODIS LAI product (i.e. MOD15A2) is one of the most widely used LAI data sources. However, the time series of MODIS LAI contains some data of low quality. For example, because of the influence of the cloud, aerosol, etc., the MODIS LAI presents the characteristics of the discontinuous in time and space. In fact, the time series of MODIS LAI include both linear and nonlinear components, which cannot be accurately modeled and predicted by either linear method or nonlinear method along. In this paper, the original LAI time series data were first smoothed with Savitzky-Golay (SG) filtration and linear interpolation; SARIMA, BP neural network and a hybrid method of SARIMA-BP neural netwok were then used for modeling and predicting MODIS LAI time series. The SARIMA-BP neural network combined both SARIMA and BP neural network, which could model the linear and the nonlinear component of MODIS LAI time series respectively. That is, the final result of SARIMA-BP neural network was the sum of the results of the two methods. Experiments showed that the time series of MODIS LAI that were smoothed with the SG filtration and linear interpolation were more smooth than original time series, with a determination coefficient up to 0.981, closer to 1 than that of SARIMA (0.941) and BP neural network (0.884); the correlation coefficient between SARIMA-BP neural network and the observation is 0.991, higher than that of between SARIMA (0.971) or BP neural network (0.942) SARIMA and the observation. Thus, it can be concluded that, the proposed SARIMA-BP neural network method can better adapt to the LAI time series, and it outperforms the SARIMA and BP neural network methods.

Unsupervised image classification is an important means to obtain land use/cover information in the field of remote sensing, since it does not require initial knowledge (training samples) for classification. Traditional methods such as k-means and ISODATA have limitations in solving this NP-hard unsupervised classification problem, mainly due to their strict assumptions about the data distribution. The bee colony optimization (BCO) is a new type of swarm intelligence, based upon which a simple and novel unsupervised bee colony optimization (UBCO) method is proposed for remote sensing image classification. UBCO possesses powerful exploitation and exploration capacities that are carried out by employed bees, onlookers and scouts. This enables the promising regions to be globally searched quickly and thoroughly, without becoming trapped on local optima. In addition, it has no restrictions on data distribution, and thus is especially suitable for handling complex remote sensing data. We tested the method on the Zhalong National Nature Reserve (ZNNR)—a typical inland wetland ecosystem in China, whose landscape is heterogeneous. The preliminary results showed that UBCO (overall accuracy = 80.81%) achieved statistically significant better classification result (McNemar test) in comparison with traditional k-means (63.11%) and other intelligent clustering methods built on genetic algorithm (UGA, 71.49%), differential evolution (UDE, 77.57%) and particle swarm optimization (UPSO, 69.86%). The robustness and superiority of UBCO were also demonstrated from the two other study sites next to the ZNNR with distinct landscapes (urban and natural landscapes). Enabling to consistently find the optimal or nearly optimal global solution in image clustering, the UBCO is thus suggested as a robust method for unsupervised remote sensing image classification, especially in the case of heterogeneous areas.

The number of clusters (i.e., the number of classes) for unsupervised classification has been recognized as an important part of remote sensing image clustering analysis. The number of classes is usually determined by cluster validity indices (CVIs). Although many CVIs have been proposed, few studies have compared and evaluated their effectiveness on remote sensing datasets. In this paper, the performance of 16 representative and commonly-used CVIs was comprehensively tested by applying the fuzzy c-means (FCM) algorithm to cluster nine types of remote sensing datasets, including multispectral (QuickBird, Landsat TM, Landsat ETM+, FLC1, and GaoFen-1) and hyperspectral datasets (Hyperion, HYDICE, ROSIS, and AVIRIS). The preliminary experimental results showed that most CVIs, including the commonly used DBI (Davies-Bouldin index) and XBI (Xie-Beni index), were not suitable for remote sensing images (especially for hyperspectral images) due to significant between-cluster overlaps; the only effective index for both multispectral and hyperspectral data sets was the WSJ index (WSJI). Such important conclusions can serve as a guideline for future remote sensing image clustering applications.