Search

Spectral information and backscatter information are both exclusively important bases for land cover classification, and these two kinds of information are found in multispectral images and SAR images, respectively. Therefore, the fusion of complementary information of multispectral and SAR images can effectively improve land cover classification accuracy. However, the existing fusion methods of multispectral and SAR images generally have some problems, such as insensitivity to edge information, serious interference by speckle noise, and unreasonable settings of fusion rules, which lead to unsatisfactory results of land cover classification. To solve this issue, a fusion method based on adaptive multiscale Gaussian co-occurrence filtering decomposition is proposed. Gaussian filtering and adaptive co-occurrence filtering are applied to the original image to smooth out speckle noise and interference edges within the textures while preserving edge information between textures. Through multi-scale spatial decomposition, the separation of detail information, edge information and basic information is realized, while multi-layer fusion of image features is performed. Finally, the fused image with low noise interference, clear boundary and uniform pixel convergence is generated. Experimental results show that the proposed method generally performs the best in eight evaluation indexes compared with ten other methods. The overall accuracy, average accuracy and Kappa coefficient of land cover classification are increased by 7.674%, 6.776%, and 0.098, respectively, compared with those of the original multispectral image in Area 1, and by 6.904%, 7.649%, and 0.089, respectively, compared with those of the original multispectral image in Area 2.

Accurate mapping of global urban man-made objects such as buildings and roads is critical for monitoring urbanization. Remote sensing imagery provides a cost-effective way of mapping these objects, but the challenge of “knowledge forgetting” arises due to urban diversity and the continuous growth of global samples. Although the existing knowledge distillation approaches can transfer knowledge from a larger teacher model to a smaller student model by distilling the knowledge learned from reliable labels, they fail to work for global-scale mapping, which lies in two aspects: low-quality labeling and fixed-size models. In this paper, we propose GUMONet, which is a semi-supervised knowledge distillation framework for global-scale urban man-made object mapping. For the first phase, a label diversity progressive learning module is introduced for generating high-quality labels in a semi-supervised manner. Label diversity is used to measure the diverse urban patterns based on spatial-semantic uncertainty, where the diversified labels clustered in object boundaries and heterogeneous areas are attributed to high spatial uncertainty and semantic uncertainty, respectively. Based on the label diversity, the model decision boundary is progressively determined from coarse to fine. Specifically, at the early stage, instances away from the decision boundary are selected to ensure the stability of the model training. As the iteration progresses, instances close to the decision boundary are associated with a higher probability of further enhancing the quality of the uncertain labels by hard sample mining. For the second phase, a size-variable knowledge distillation module is adopted to optimize the data-model matching process. This module consists of a noise teacher model that prevents overfitting by injecting noise perturbations to increase the data distribution complexity and a size-variable student model that avoids underfitting by dynamically adjusting its size with the growth of global samples. We applied GUMONet to six study areas across four continents, with data from different sensors, achieving an 18.97% improvement in intersection over union, compared with the previous methods. Our results also demonstrate a positive correlation between urban development and urban diversity, with a correlation coefficient of 0.749. As urban development progresses, urban diversity stabilizes and building transformation becomes the primary means of promoting further development.

The spatial resolution of land cover mapping has been increasing with the evolution of Earth observation technology. However, the higher spatial resolution makes it more laborious to collect training samples for efficient land-cover product updating. Fortunately, the existing historical products with a lower spatial resolution can be used as labels to achieve cross-resolution mapping with the latest images of a higher resolution. Although cross-resolution mapping can generate a large number of low-cost training labels, the labels can be noisy due to resolution mismatch or semantic errors. Furthermore, the existing deep learning based classification models have difficulty in maintaining the output resolution. In this article, a cross-resolution land-cover mapping framework with noisy label learning is proposed to complete high-resolution land-cover mapping based on noisy lower-resolution labels. The proposed method contains three parts: 1) To solve the resolution mismatch problem, the training labels are refined by modeling the spectral similarity and the spatial adjacency between the labels and the images using a conditional random field model. 2) To avoid the problem of resolution loss with the deep fully convolutional neural networks, a high-resolution deep semantic segmentation network is proposed to achieve deep feature extraction while maintaining the output resolution. 3) To eliminate the influence of semantic errors in the generated labels, a class-conditional label correction method is proposed, which detects and corrects the abnormal incorrect labels to facilitate network training. A national-scale land-cover mapping experiment for China was carried out. The 10-m spatial resolution land-cover map of China for 2020 was produced with improved accuracy based on the 30-m spatial resolution historical product, which shows the practicability of the proposed CRLC for rapid land-cover mapping.

Benefiting from the development of deep learning, researchers have made significant progress and achieved superior performance in the semantic segmentation of remote sensing (RS) data. However, when encountering an unseen scenario, the performance of a trained model deteriorates dramatically because of the domain shift. Unsupervised domain adaptation (UDA) provides an alternative to address the issue. Aligning the high-level representations via adversarial learning is a popular way, but it is difficult when there is a large gap in input space. With this consideration, we design a framework to jointly align the distribution in input and feature space. For input space alignment, we unify the resolution for the consistency of content and propose a lightweight module named Digital Number Transformer (DNT) to reduce the visual differences. For feature space alignment, we design a Multi-Scale Feature aggregation (MSF) module and introduce the Fine-Grained Discriminator (FGD) to conduct a category-level alignment at multiple layers so that features can be fully aligned and negative transfer can be reduced. We carried out experiments in diverse cross-domain scenarios, including the discrepancy in geographic position and the discrepancy in both geographic position and imaging mode. Comprehensive experiments demonstrate that our method outperforms other state-of-the-art methods in all scenarios.

Urban waterlogging probability assessment is critical to emergency response and policymaking. Remote Sensing (RS) is a rich and reliable data source for waterlogging monitoring and evaluation through water body extraction derived from the pre- and post-disaster RS images. However, RS images are usually limited to the revisit cycle and cloud cover. To solve this issue, social media data have been considered as another data source which are immune to the weather such as clouds and can reflect the real-time public response for disaster, which leads itself a compensation for RS images. In this paper, we propose a coarse-to-fine waterlogging probability assessment framework based on multisource data including real-time social media data, near real-time RS image and historical geographic information, in which a coarse waterlogging probability map is refined by using the real-time information extracted from social media data to acquire a more accurate waterlogging probability. Firstly, to generate a coarse waterlogging probability map, the historical inundated areas are derived from Digital Elevation Model (DEM) and historical waterlogging points, then the geographic features are extracted from DEM and RS image, which will be input to a Random Forest (RF) classifier to estimate the likelihood of hazards. Secondly, the real-time waterlogging-related information is extracted from social media data, where the Convolutional Neural Network (CNN) model is applied to exploit the semantic information of sentences by capturing the local and position-invariant features using convolution kernel. Finally, fine waterlogging probability map scan be generated based on morphological method, in which real-time waterlogging-related social media data are taken as isolated highlight point and used to refine the coarse waterlogging probability map by a gray dilation pattern considering the distance-decay effect. The 2016 Wuhan waterlogging and 2018 Chengdu waterlogging are taken as case studies to demonstrate the effectiveness of the proposed framework. It can be concluded from the results that by integrating RS image and social media data, more accurate waterlogging probability maps can be generated, which can be further applied for inundated areas identification and disaster monitoring.

The recognition of fine-grained objects is crucial for future remote sensing applications, but this task is faced with the few-shot problem due to limited labeled data. In addition, the existing few-shot learning methods do not consider the unique characteristics of remote sensing objects, i.e., the complex backgrounds and the difficulty of extracting fine-grained features, leading to suboptimal performance. In this study, we developed an improved task sampling strategy for few-shot learning that optimizes the target distribution. The proposed approach incorporates broad category information, where each sample is assigned both a broad and fine category label and converts the target task distribution into a fine-grained distribution. This ensures that the model focuses on extracting fine-grained features for the corresponding broad category. We also introduce a category generation method that ensures the same number of fine-grained categories in each task to improve the model accuracy. The experimental results demonstrate that the proposed strategy outperforms the existing object recognition methods. We believe that this strategy has the potential to be applied to fine-grained few-shot object recognition, thus contributing to the development of high-precision remote sensing applications.

Face inpainting aims to repaired damaged images caused by occlusion or cover. In recent years, deep learning based approaches have shown promising results for the challenging task of image inpainting. However, there are still limitation in reconstructing reasonable structures because of over-smoothed and/or blurred results. The distorted structures or blurred textures are inconsistent with surrounding areas and require further post-processing to blend the results. In this paper, we present a novel generative model-based approach, which consisted by nested two Generative Adversarial Networks (GAN), the sub-confrontation GAN in generator and parent-confrontation GAN. The sub-confrontation GAN, which is in the image generator of parent-confrontation GAN, can find the location of missing area and reduce mode collapse as a prior constraint. To avoid generating vague details, a novel residual structure is designed in the sub-confrontation GAN to deliver richer original image information to the deeper layers. The parent-confrontation GAN includes an image generation part and a discrimination part. The discrimination part of parent-confrontation GAN includes global and local discriminator, which benefits the reconstruction of overall coherency of the repaired image while obtaining local details. The experiments are executed over the publicly available dataset CelebA, and the results show that our method outperforms current state-of-the-art techniques quantitatively and qualitatively.

The nonlinear radiation distortions (NRD) among multimodal remote sensing images bring enormous challenges to image registration. The traditional feature-based registration methods commonly use the image intensity or gradient information to detect and describe the features that are sensitive to NRD. However, the nonlinear mapping of the corresponding features of the multimodal images often results in failure of the feature matching, as well as the image registration. In this paper, a modality-free multimodal remote sensing image registration method (SRIFT) is proposed for the registration of multimodal remote sensing images, which is invariant to scale, radiation, and rotation. In SRIFT, the nonlinear diffusion scale (NDS) space is first established to construct a multi-scale space. A local orientation and scale phase congruency (LOSPC) algorithm are then used so that the features of the images with NRD are mapped to establish a one-to-one correspondence, to obtain sufficiently stable key points. In the feature description stage, a rotation-invariant coordinate (RIC) system is adopted to build a descriptor, without requiring estimation of the main direction. The experiments undertaken in this study included one set of simulated data experiments and nine groups of experiments with different types of real multimodal remote sensing images with rotation and scale differences (including synthetic aperture radar (SAR)/optical, digital surface model (DSM)/optical, light detection and ranging (LiDAR) intensity/optical, near-infrared (NIR)/optical, short-wave infrared (SWIR)/optical, classification/optical, and map/optical image pairs), to test the proposed algorithm from both quantitative and qualitative aspects. The experimental results showed that the proposed method has strong robustness to NRD, being invariant to scale, radiation, and rotation, and the achieved registration precision was better than that of the state-of-the-art methods.

Vertical deformations caused by non-tidal mass variations still remain in global navigation satellite system (GNSS) height time series, and can be computed from both Gravity Recovery and Climate Experiment (GRACE) and geophysical models. In this study, we provide a thorough evaluation of the relationships between these different techniques in the global scale by comparing non-tidal vertical deformations from IGS second reprocessing campaign (IG2), GRACE and Global Geophysical Fluid Center (GGFC) solutions, and investigate the noise properties of the GNSS corrected by GRACE solutions and GNSS corrected by GGFC solutions for global stations using optimal noise models. Our results demonstrate that the consistency between seasonal vertical deformations derived from GNSS, GRACE and GGFC is high. When correcting GNSS deformations with GRACE and GGFC solutions, 81% and 73% of the 186 stations have the weight root mean square (WRMS) reduction, respectively. The WRMS variations averaged over all stations are −12.3% and −5.6%, respectively for GNSS corrected by GRACE and GNSS corrected by GGFC solutions. The obvious difference occurs in the GNSS corrected by GGFC solutions WRMS increase, with the mean increase value up to 29%, mainly happening to stations located on islands or small peninsulas. In addition, noise properties of the GNSS corrected by GRACE solutions and GNSS corrected by GGFC solutions for global stations are investigated using optimal noise models. After correcting non-tidal loading effects, the solutions of GNSS corrected by GRACE solutions have the lowest noise level, and can occupy 5% of the noise behavior presenting in global stations, while the solutions of GNSS corrected by GGFC solutions can bring more than 5% of the noise into global stations, implying that GRACE correction solutions can present more favorable results when interpreting GNSS non-tidal loading deformations.

Change detection (CD) of natural land cover is important for environmental protection and to maintain an ecological balance. The Landsat series of satellites provide continuous observation of the Earth’s surface and is sensitive to reflection of water, soil and vegetation. It offers fine spatial resolutions (15–80 m) and short revisit times (16–18 days). Therefore, Landsat imagery is suitable for monitoring natural land cover changes. Clustering-based CD methods using evolutionary algorithms (EAs) can be applied to Landsat images to obtain optimal changed and unchanged clustering centers (clusters) with minimum clustering index. However, they directly analyze difference image (DI), which finds itself subject to interference by Gaussian noise and local brightness distortion in Landsat data, resulting in false alarms in detection results. In order to reduce image interferences and improve CD accuracy, we proposed an unsupervised CD method based on multi-feature clustering using the differential evolution algorithm (M-DECD) for Landsat Imagery. First, according to characteristics of Landsat data, a multi-feature space is constructed with three elements: Wiener de-noising, detail enhancement, and structural similarity. Then, a CD method based on differential evolution (DE) algorithm and fuzzy clustering is proposed to obtain global optimal clusters in the multi-feature space, and generate a binary change map (CM). In addition, the control parameters of the DE algorithm are adjusted to improve the robustness of M-DECD. The experimental results obtained with four Landsat datasets confirm the effectiveness of M-DECD. Compared with the results of conventional methods and the current state-of-the-art methods based on evolutionary clustering, the detection accuracies of the M-DECD on the Mexico dataset and the Sardinia dataset are very close to the best results. The accuracies of the M-DECD in the Alaska dataset and the large Canada dataset increased by about 3.3% and 11.9%, respectively. This indicates that multiple features are suitable for Landsat images and the DE algorithm is effective in searching for an optimal CD result.

Hyperspectral images and light detection and ranging (LiDAR) data have, respectively, the high spectral resolution and accurate elevation information required for urban land-use/land-cover (LULC) classification. To combine the respective advantages of hyperspectral and LiDAR data, this paper proposes an optimal decision fusion method based on adaptive differential evolution, namely ODF-ADE, for urban LULC classification. In the ODF-ADE framework the normalized difference vegetation index (NDVI), gray-level co-occurrence matrix (GLCM) and digital surface model (DSM) are extracted to form the feature map. The three different classifiers of the maximum likelihood classifier (MLC), support vector machine (SVM) and multinomial logistic regression (MLR) are used to classify the extracted features. To find the optimal weights for the different classification maps, weighted voting is used to obtain the classification result and the weights of each classification map are optimized by the differential evolution algorithm which uses a self-adaptive strategy to obtain the parameter adaptively. The final classification map is obtained after post-processing based on conditional random fields (CRF). The experimental results confirm that the proposed algorithm is very effective in urban LULC classification.

Remote sensing image clustering is a challenging task considering its intrinsic complexity. Recently, by combining the spectral and spatial information of the remote sensing data, the clustering performance can be dramatically enhanced, termed as Spectral-Spatial Clustering (SSC). However, it has always been difficult to determine the weight parameter for balancing the spectral term and spatial term of the clustering objective function. In this paper, spectral-spatial clustering with a local weight parameter determination method for remote sensing image was proposed, i.e., L-SSC. In L-SSC, considering the large scale of remote sensing images, the weight parameter can be determined locally in a patch image instead of the whole image. Afterwards, the local weight parameter was used in constructing the objective function of L-SSC. Thus, the remote sensing image clustering problem was transformed into an optimization problem. Finally, in order to achieve a better optimization performance, a variant of differential evolution (i.e., jDE) was used as the optimizer due to its powerful optimization capability. Experimental results on three remote sensing images, including a Wuhan TM image, a Fancun Quickbird image, and an Indian Pine AVIRIS image, demonstrated that the proposed L-SSC can acquire higher clustering accuracy in comparison to other spectral-spatial clustering methods.