Search

This document is a summary of a special issue on advanced machine learning and deep learning techniques for remote sensing. The issue includes 16 research papers that cover a range of topics, including hyperspectral image classification, moving point target detection, radar echo extrapolation, and remote sensing object detection. Each paper introduces a novel approach or model and provides extensive testing and evaluation to demonstrate its effectiveness. The insights shared in this special issue are expected to contribute to future advancements in artificial intelligence-based remote sensing research. [Extracted from the article]

Deep learning has gained widespread interest in the task of building semantic segmentation modelling using remote sensing images; however, neural network models require a large number of training samples to achieve better classification performance, and the models are more sensitive to error patches in the training samples. The training samples obtained in semi-supervised classification methods need less reliable weakly labelled samples, but current semi-supervised classification research puts the generated weak samples directly into the model for applications, with less consideration of the impact of the accuracy and quality improvement of the weak samples on the subsequent model classification. Therefore, to address the problem of generating and optimising the quality of weak samples from training data in deep learning, this paper proposes a semi-supervised building classification framework. Firstly, based on the test results of the remote sensing image segmentation model and the unsupervised classification results of LiDAR point cloud data, this paper quickly generates weak image samples of buildings. Secondly, in order to improve the quality of the spots of the weak samples, an iterative optimisation strategy of the weak samples is proposed to compare and analyse the weak samples with the real samples and extract the accurate samples from the weak samples. Finally, the real samples, the weak samples, and the optimised weak samples are input into the semantic segmentation model of buildings for accuracy evaluation and analysis. The effectiveness of this paper's approach was experimentally verified on two different building datasets, and the optimised weak samples improved by 1.9% and 0.6%, respectively, in the test accuracy mIoU compared to the initial weak samples. The results demonstrate that the semi-supervised classification framework proposed in this paper can be used to alleviate the model's demand for a large number of real-labelled samples while improving the ability to utilise weak samples, and it can be used as an alternative to fully supervised classification methods in deep learning model applications that require a large number of training samples. [ABSTRACT FROM AUTHOR]

Mussel platforms are big floating structures made of wood (normally about 20 m × 20 m or even a bit larger) that are used for aquaculture. They are used for supporting the growth of mussels in suitable marine waters. These structures are very common near the Galician coastline. For their maintenance and tracking, it is quite convenient to be able to produce a periodic census of these structures, including their current count and position. Images from Earth observation satellites are, a priori, a convenient choice for this purpose. This paper describes an application capable of automatically supporting such a census using optical images taken at different wavelength intervals. The images are captured by the two Sentinel 2 satellites (Sentinel 2A and Sentinel 2B, both from the Copernicus Project). The Copernicus satellites are run by the European Space Agency, and the produced images are freely distributed on the Internet. Sentinel 2 images include thirteen frequency bands and are updated every five days. In our proposal, remote-sensing normalized (differential) indexes are used, and machine-learning techniques are applied to multiband data. Different methods are described and tested. The results obtained in this paper are satisfactory and prove the approach is suitable for the intended purpose. In conclusion, it is worth noting that artificial neural networks turn out to be particularly good for this problem, even with a moderate level of complexity in their design. The developed methodology can be easily re-used and adapted for similar marine environments. [ABSTRACT FROM AUTHOR]

Forest biomass estimation is undoubtedly one of the most pressing research subjects at present. Combining multi-source remote sensing information can give full play to the advantages of different remote sensing technologies, providing more comprehensive and rich information for aboveground biomass (AGB) estimation research. Based on Landsat 8, Sentinel-2A, and ALOS2 PALSAR data, this paper takes the artificial coniferous forests in the Saihanba Forest of Hebei Province as the object of study, fully explores and establishes remote sensing factors and information related to forest structure, gives full play to the advantages of spectral signals in detecting the horizontal structure and multi-dimensional synthetic aperture radar (SAR) data in detecting the vertical structure, and combines environmental factors to carry out multivariate synergistic methods of estimating the AGB. This paper uses three variable selection methods (Pearson correlation coefficient, random forest significance, and the least absolute shrinkage and selection operator (LASSO)) to establish the variable sets, combining them with three typical non-parametric models to estimate AGB, namely, random forest (RF), support vector regression (SVR), and artificial neural network (ANN), to analyze the effect of forest structure on biomass estimation, explore the suitable AGB of artificial coniferous forests estimation of machine learning models, and develop the method of quantifying saturation value of the combined variables. The results show that the horizontal structure is more capable of explaining the AGB compared to the vertical structure information, and that combining the multi-structure information can improve the model results and the saturation value to a great extent. In this study, different sets of variables can produce relatively superior results in different models. The variable set selected using LASSO gives the best results in the SVR model, with an R 2 values of 0.9998 and 0.8792 for the training and the test set, respectively, and the highest saturation value obtained is 185.73 t/ha, which is beyond the range of the measured data. The problem of saturation in biomass estimation in boreal medium- and high-density forests was overcome to a certain extent, and the AGB of the Saihanba area was better estimated. [ABSTRACT FROM AUTHOR]

The paper demonstrates the effectiveness of the SNOWED dataset, specifically designed for identifying water bodies in Sentinel-2 images, in developing a remote sensing system based on deep neural networks. For this purpose, a system is implemented for monitoring the Po River, Italy's most important watercourse. By leveraging the SNOWED dataset, a simple U-Net neural model is trained to segment satellite images and distinguish, in general, water and land regions. After verifying its performance in segmenting the SNOWED validation set, the trained neural network is employed to measure the area of water regions along the Po River, a task that involves segmenting a large number of images that are quite different from those in SNOWED. It is clearly shown that SNOWED-based water area measurements describe the river status, in terms of flood or drought periods, with a surprisingly good accordance with water level measurements provided by 23 in situ gauge stations (official measurements managed by the Interregional Agency for the Po). Consequently, the sensing system is used to take measurements at 100 "virtual" gauge stations along the Po River, over the 10-year period (2015–2024) covered by the Sentinel-2 satellites of the Copernicus Programme. In this way, an overall space-time monitoring of the Po River is obtained, with a spatial resolution unattainable, in a cost-effective way, by local physical sensors. Altogether, the obtained results demonstrate not only the usefulness of the SNOWED dataset for deep learning-based satellite sensing, but also the ability of such sensing systems to effectively complement traditional in situ sensing stations, providing precious tools for environmental monitoring, especially of locations difficult to reach, and permitting the reconstruction of historical data related to floods and draughts. Although physical monitoring stations are designed for rapid monitoring and prevention of flood or other disasters, the developed tool for remote sensing of water bodies could help decision makers to define long-term policies to reduce specific risks in areas not covered by physical monitoring or to define medium- to long-term strategies such as dam construction or infrastructure design. [ABSTRACT FROM AUTHOR]

In remote sensing scene classification (RSSC), restrictions on real-time processing on power consumption, performance, and resources necessitate the compression of neural networks. Unlike artificial neural networks (ANNs), spiking neural networks (SNNs) convey information through spikes, offering superior energy efficiency and biological plausibility. However, the high latency of SNNs restricts their practical application in RSSC. Therefore, there is an urgent need to research ultra-low-latency SNNs. As latency decreases, the performance of the SNN significantly deteriorates. To address this challenge, we propose a novel spatio-temporal pruning method that enhances the feature capture capability of ultra-low-latency SNNs. Our approach integrates spatial fundamental structures during the training process, which are subsequently pruned. We conduct a comprehensive evaluation of the impacts of these structures across classic network architectures, such as VGG and ResNet, demonstrating the generalizability of our method. Furthermore, we develop an ultra-low-latency training framework for SNNs to validate the effectiveness of our approach. In this paper, we successfully achieve high-performance ultra-low-latency SNNs with a single time step for the first time in RSSC. Remarkably, our SNN with one time step achieves at least 200 times faster inference time while maintaining a performance comparable to those of other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The integration of multi-source remote sensing data, bolstered by advancements in deep learning, has emerged as a pivotal strategy for enhancing land use and land cover (LULC) classification accuracy. However, current methods often fail to consider the numerous prior knowledge of remote sensing images and the characteristics of heterogeneous remote sensing data, resulting in data loss between different modalities and the loss of a significant amount of useful information, thus affecting classification accuracy. To tackle these challenges, this paper proposes a LULC classification method based on remote sensing data that combines a Transformer and cross-pseudo-siamese learning deep neural network (TCPSNet). It first conducts shallow feature extraction in a dynamic multi-scale manner, fully leveraging the prior information of remote sensing data. Then, it further models deep features through the multimodal cross-attention module (MCAM) and cross-pseudo-siamese learning module (CPSLM). Finally, it achieves comprehensive fusion of local and global features through feature-level fusion and decision-level fusion combinations. Extensive experiments on datasets such as Trento, Houston 2013, Augsburg, MUUFL and Berlin demonstrate the superior performance of the proposed TCPSNet. The overall accuracy (OA) of the network on the Trento, Houston 2013 and Augsburg datasets is of 99.76%, 99.92%, 97.41%, 87.97% and 97.96%, respectively. [ABSTRACT FROM AUTHOR]

Soil moisture is an important component of the hydrologic cycle and ecosystem functioning, and it has a significant impact on agricultural production, climate change and natural disasters. Despite the availability of machine-learning techniques for estimating soil moisture from high-resolution remote-sensing imagery, including synthetic aperture radar (SAR) data and optical remote sensing, comprehensive comparative studies of these techniques remain limited. This paper addresses this gap by systematically comparing the performance of four tree-based ensemble-learning models (random forest (RF), extreme gradient boosting (XGBoost), light gradient-boosting machine (LightGBM), and category boosting (CatBoost)) and three deep-learning models (deep neural network (DNN), convolutional neural network (CNN), and gated recurrent unit (GRU)) in terms of soil moisture estimation. Additionally, we introduce and evaluate the effectiveness of four different stacking methods for model fusion, an approach that is relatively novel in this context. Moreover, Sentinel-1 C-band dual-polarization SAR and Sentinel-2 multispectral data, as well as NASADEM and geographical code and temporal code features, are used as input variables to retrieve the soil moisture in the ShanDian River Basin in China. Our findings reveal that the tree-based ensemble-learning models outperform the deep-learning models, with LightGBM being the best individual model, while the stacking approach can further enhance the accuracy and robustness of soil moisture estimation. Moreover, the stacking all boosting classes ensemble-learning model (SABM), which integrates only boosting-type models, demonstrates superior accuracy and robustness in soil moisture estimation. The SHAP value analysis reveals that ensemble learning can utilize more complex features than deep learning. This study provides an effective method for retrieving soil moisture using machine-learning and high-resolution remote-sensing data, demonstrating the application value of SAR data and high-resolution optical remote-sensing data in soil moisture monitoring. [ABSTRACT FROM AUTHOR]

This paper proposes a novel approach to enhance the multichannel fiber optic sensing systems by integrating an Inverse Fast Fourier Transform-based Deep Neural Network (IFFT-DNN) to accurately predict sensor responses despite signals overlapping and crosstalk between sensors. The IFFT-DNN leverages both frequency and time domain information, enabling a comprehensive feature extraction which enhances the prediction accuracy and reliability performance. To investigate the IFFT-DNN's performance, we propose a multichannel water level sensing system based on Free Space Optics (FSO) to measure the water level at multiple points in remote areas. The experimental results demonstrate the system's high precision, with a Mean Absolute Error (MAE) of 0.07 cm, even in complex conditions. Hence, this system provides a cost-effective and reliable remote water level sensing solution, highlighting its practical applicability in various industrial settings. [ABSTRACT FROM AUTHOR]

Rapid and accurate extraction of landslide areas triggered by earthquakes has far-reaching significance for geological disaster risk assessment and emergency rescue. At present, visual interpretation and field survey are still the most-commonly used methods for landslide identification, but these methods are often time-consuming and costly. For this reason, this paper tackles the problem of co-seismic landslide identification and the fact that there is little sample information in existing studies on landslide. A landslide sample dataset with 4000 tags was produced. With the YOLOv3 algorithm as the core, a convolutional neural network model with landslide characteristics was established to automatically recognize co-seismic landslides in satellite remote sensing images. By comparing it with the graphical interpretation results of remote sensing images, we found that the remote sensing for landslide recognition model constructed in this paper demonstrated high recognition accuracy and fast speed. The F1 value was 0.93, indicating that the constructed model was stable. The research results can provide reference for emergency rescue and disaster investigation of the same co-seismic landslide disaster. [ABSTRACT FROM AUTHOR]

Water bodies can offer local microclimates that have the potential to attenuate the effects of urban heat islands by reducing local temperature. This capability is shaded when the river is channelized. In such cases, the river temperature rises during hot periods, leading to negative impacts on the water quality. The main aim of this paper is to develop a local simple model to predict the temperature of the Dâmbovița River at its exit from Bucharest City, the capital of Romania. The location is chosen based on the historical critical impacts, in terms of extreme heatwaves that took place during hot summers, as well as future possible risks due to climate change. The water temperature prediction model is based on an artificial neural network that uses the Levenberg–Marquardt algorithm, due to its stability and rapid convergence capabilities. The model forecasts, with an accuracy of ±1 °C, the water temperature in an ungauged, downstream location, as a function of measured air and upstream water temperatures. The proposed model represents a first attempt to provide water managers in Bucharest City with a useful tool that will allow them to take timely measures to counteract the unwanted effects that can be generated by high water temperatures. [ABSTRACT FROM AUTHOR]

Ship classification using the synthetic aperture radar (SAR) images has a significant role in remote sensing applications. Aiming at the problems of excessive model parameters numbers and high energy consumption in the traditional deep learning methods for the SAR ship classification, this paper provides an energy-efficient SAR ship classification paradigm that combines spiking neural networks (SNNs) with Siamese network architecture, for the first time in the field of SAR ship classification, which is called the Siam-SpikingShipCLSNet. It combines the advantage of SNNs in energy consumption and the advantage of the idea in performances that use the Siamese neuron network to fuse the features from dual-polarized SAR images. Additionally, we migrated the feature fusion strategy from CNN-based Siamese neural networks to the SNN domain and analyzed the effects of various spiking feature fusion methods on the Siamese SNN. Finally, an end-to-end error backpropagation optimization method based on the surrogate gradient has been adopted to train this model. Experimental results tested on the OpenSARShip2.0 dataset have demonstrated the correctness and effectiveness of the proposed SAR ship classification strategy, which has the advantages of the higher accuracy, fewer parameters and lower energy consumption compared with the mainstream deep learning method of the SAR ship classification. [ABSTRACT FROM AUTHOR]

Recently, data-driven approaches have become the dominant solution for prediction problems in agricultural industries. Several deep learning models have been applied to crop yield prediction in smart farming. In this paper, we proposed an efficient hybrid deep learning model that coordinates the outcomes of a classification model and a regression model in deep learning via the shared layers to predict the rice crop yield. Three statistical analyses on the features, including Pearson correlation coefficients (PCC), Shapley additive explanations (SHAP), and recursive feature elimination with cross-validation (RFECV), are proposed to select the most relevant ones for the predictive goal to reduce the model training time. The data preprocessing normalizes the features of the collected data into specific ranges of values and then reformats them into a three-dimensional matrix. As a result, the root-mean-square error (RMSE) of the proposed model in rice yield prediction has achieved 344.56 and an R-squared of 0.64. The overall performance of the proposed model is better than the other deep learning models, such as the multi-parametric deep neural networks (MDNNs) (i.e., RMSE = 370.80, R-squared = 0.59) and the artificial neural networks (ANNs) (i.e., RMSE = 550.03, R-squared = 0.09). The proposed model has demonstrated significant improvement in the predictive results of distinguishing high yield from low yield with 90% accuracy and 94% F1 score. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing images (HSIs) have both spectral and spatial characteristics. The adept exploitation of these attributes is central to enhancing the classification accuracy of HSIs. In order to effectively utilize spatial and spectral features to classify HSIs, this paper proposes a method for the spatial feature extraction of HSIs based on a mutually guided image filter (muGIF) and combined with the band-distance-grouped principal component. Firstly, aiming at the problem that previously guided image filtering cannot effectively deal with the inconsistent information structure between the guided and target information, a method for extracting spatial features using muGIF is proposed. Then, aiming at the problem of the information loss caused by a single principal component as a guided image in the traditional GIF-based spatial–spectral classification, a spatial feature-extraction framework based on the band-distance-grouped principal component is proposed. The method groups the bands according to the band distance and extracts the principal components of each set of band subsets as the guide map of the current band subset to filter the HSIs. A deep convolutional neural network model and a generative adversarial network model for the filtered HSIs are constructed and then trained using samples for HSIs' spatial–spectral classification. Experiments show that compared with the traditional methods and several popular spatial–spectral HSI classification methods based on a filter, the proposed methods based on muGIF can effectively extract the spatial–spectral features and improve the classification accuracy of HSIs. [ABSTRACT FROM AUTHOR]

Optical remote sensing videos, as a new source of remote sensing data that has emerged in recent years, have significant potential in remote sensing applications, especially national defense. In this paper, a tracking pipeline named TDNet (tracking while detecting based on a neural network) is proposed for optical remote sensing videos based on a correlation filter and deep neural networks. The pipeline is used to simultaneously track ships and planes in videos. There are many target tracking methods for general video data, but they suffer some difficulties in remote sensing videos with low resolution and those influenced by weather conditions. The tracked targets are usually misty. Therefore, in TDNet, we propose a new multi-target tracking method called MT-KCF and a detecting-assisted tracking (i.e., DAT) module to improve tracking accuracy and precision. Meanwhile, we also design a new target recognition (i.e., NTR) module to recognise newly emerged targets. In order to verify the performance of TDNet, we compare our method with several state-of-the-art tracking methods on optical video remote sensing data sets acquired from the Jilin No. 1 satellite. The experimental results demonstrate the effectiveness and the state-of-the-art performance of the proposed method. The proposed method can achieve more than 90% performance in terms of precision for single-target tracking tasks and more than 85% performance in terms of MOTA for multi-object tracking tasks. [ABSTRACT FROM AUTHOR]

As deep neural networks (DNNs) are widely used in the field of remote sensing image recognition, there is a model security issue that cannot be ignored. DNNs have been shown to be vulnerable to small perturbations in a large number of studies in the past, and this security risk naturally exists in remote sensing object detection models based on DNNs. The complexity of remote sensing object detection models makes it difficult to implement adversarial attacks on them, resulting in the current lack of systematic research on adversarial examples in the field of remote sensing image recognition. In order to better deal with the adversarial threats that remote sensing image recognition models may confront and to provide an effective means for evaluating the robustness of the models, this paper takes the adversarial examples for remote sensing image recognition as the research goal and systematically studies vanishing attacks against a remote sensing image object detection model. To solve the problem of difficult attack implementation on remote sensing image object detection, adversarial attack adaptation methods based on interpolation scaling and patch perturbation stacking are proposed in this paper, which realizes the adaptation of classical attack algorithms. We propose a hot restart perturbation update strategy and the joint attack of the first and second stages of the two-stage remote sensing object detection model is realized through the design of the attack loss function. For the problem of the modification cost of global pixel attack being too large, a local pixel attack algorithm based on sensitive pixel location is proposed in this paper. By searching the location of the sensitive pixels and constructing the mask of attack area, good local pixel attack effect is achieved. Experimental results show that the average pixel modification rate of the proposed attack method decreases to less than 4% and the vanishing rate can still be maintained above 80%, which effectively achieves the balance between attack effect and attack cost. [ABSTRACT FROM AUTHOR]

Remote sensing change detection involves detecting pixels that have changed from a bi-temporal image of the same location. Current mainstream change detection models use encoder-decoder structures as well as Siamese networks. However, there are still some challenges with this: (1) Existing change feature fusion approaches do not take into account the symmetry of change features, which leads to information loss; (2) The encoder is independent of the change detection task, and feature extraction is performed separately for dual-time images, which leads to underutilization of the encoder parameters; (3) There are problems of unbalanced positive and negative samples and bad edge region detection. To solve the above problems, a mutual feature-aware network (MFNet) is proposed in this paper. Three modules are proposed for the purpose: (1) A symmetric change feature fusion module (SCFM), which uses double-branch feature selection without losing feature information and focuses explicitly on focal spatial regions based on cosine similarity to introduce strong a priori information; (2) A mutual feature-aware module (MFAM), which introduces change features in advance at the encoder stage and uses a cross-type attention mechanism for long-range dependence modeling; (3) A loss function for edge regions. After detailed experiments, the F1 scores of MFNet on SYSU-CD and LEVIR-CD were 83.11% and 91.52%, respectively, outperforming several advanced algorithms, demonstrating the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Marine oil spills can cause serious damage to marine ecosystems and biological species, and the pollution is difficult to repair in the short term. Accurate oil type identification and oil thickness quantification are of great significance for marine oil spill emergency response and damage assessment. In recent years, hyperspectral remote sensing technology has become an effective means to monitor marine oil spills. The spectral and spatial features of oil spill images at different levels are different. To accurately identify oil spill types and quantify oil film thickness, and perform better extraction of spectral and spatial features, a multilevel spatial and spectral feature extraction network is proposed in this study. First, the graph convolutional neural network and graph attentional neural network models were used to extract spectral and spatial features in non-Euclidean space, respectively, and then the designed modules based on 2D expansion convolution, depth convolution, and point convolution were applied to extract feature information in Euclidean space; after that, a multilevel feature fusion method was developed to fuse the obtained spatial and spectral features in Euclidean space in a complementary way to obtain multilevel features. Finally, the multilevel features were fused at the feature level to obtain the oil spill information. The experimental results show that compared with CGCNN, SSRN, and A2S2KResNet algorithms, the accuracy of oil type identification and oil film thickness classification of the proposed method in this paper is improved by 12.82%, 0.06%, and 0.08% and 2.23%, 0.69%, and 0.47%, respectively, which proves that the method in this paper can effectively extract oil spill information and identify different oil spill types and different oil film thicknesses. [ABSTRACT FROM AUTHOR]

As a safety-related application, visual systems based on deep neural networks (DNNs) in modern unmanned aerial vehicles (UAVs) show adversarial vulnerability when performing real-time inference. Recently, deep ensembles with various defensive strategies against adversarial samples have drawn much attention due to the increased diversity and reduced variance for their members. Aimed at the recognition task of remote sensing images (RSIs), this paper proposes to use a reactive-proactive ensemble defense framework to solve the security problem. In reactive defense, we fuse scoring functions of several classical detection algorithms with the hidden features and average output confidences from sub-models as a second fusion. In terms of proactive defense, we attempt two strategies, including enhancing the robustness of each sub-model and limiting the transferability among sub-models. In practical applications, the real-time RSIs are first input to the reactive defense part, which can detect and reject the adversarial RSIs. The accepted ones are then passed to robust recognition with a proactive defense. We conduct extensive experiments on three benchmark RSI datasets (i.e., UCM, AID, and FGSC-23). The experimental results show that the deep ensemble method of reactive and proactive defense performs very well in gradient-based attacks. The analysis of the applicable attack scenarios for each proactive ensemble defense is also helpful for this field. We also perform a case study with the whole framework in the black-box scenario, and the highest detection rate reaches 93.25%. Most of the adversarial RSIs can be rejected in advance or correctly recognized by the enhanced deep ensemble. This article is the first one to combine reactive and proactive defenses with a deep ensemble against adversarial attacks in the context of RSI recognition for DNN-based UAVs. [ABSTRACT FROM AUTHOR]

Deep neural networks have made great achievements in remote sensing image analyses; however, previous studies have shown that deep neural networks exhibit incredible vulnerability to adversarial examples, which raises concerns about regional safety and production safety. In this paper, we propose an adversarial denoising method based on latent representation guidance for remote sensing image scene classification. In the training phase, we train a variational autoencoder to reconstruct the data using only the clean dataset. At test time, we first calculate the normalized mutual information between the reconstructed image using the variational autoencoder and the reference image as denoised by a discrete cosine transform. The reconstructed image is selectively utilized according to the result of the image quality assessment. Then, the latent representation of the current image is iteratively updated according to the reconstruction loss so as to gradually eliminate the influence of adversarial noise. Because the training of the denoiser only involves clean data, the proposed method is more robust against unknown adversarial noise. Experimental results on the scene classification dataset show the effectiveness of the proposed method. Furthermore, the method achieves better robust accuracy compared with state-of-the-art adversarial defense methods in image classification tasks. [ABSTRACT FROM AUTHOR]

Featured Application: This paper introduces a neural-network-based landscape search engine tool for the state of Wisconsin. It provides several examples of how the application works and suggests avenues for future research. The task of image retrieval is common in the world of data science and deep learning, but it has received less attention in the field of remote sensing. The authors seek to fill this gap in research through the presentation of a web-based landscape search engine for the US state of Wisconsin. The application allows users to select a location on the map and to find similar locations based on terrain and vegetation characteristics. It utilizes three neural network models—VGG16, ResNet-50, and NasNet—on digital elevation model data, and uses the NDVI mean and standard deviation for comparing vegetation data. The results indicate that VGG16 and ResNet50 generally return more favorable results, and the tool appears to be an important first step toward building a more robust, multi-input, high resolution landscape search engine in the future. The tool, called LSE Wisconsin, is hosted publicly on ShinyApps.io. [ABSTRACT FROM AUTHOR]

The extraction and classification of crops is the core issue of agricultural remote sensing. The precise classification of crop types is of great significance to the monitoring and evaluation of crops planting area, growth, and yield. Based on the Google Earth Engine and Google Colab cloud platform, this study takes the typical agricultural oasis area of Xiangride Town, Qinghai Province, as an example. It compares traditional machine learning (random forest, RF), object-oriented classification (object-oriented, OO), and deep neural networks (DNN), which proposes a random forest combined with deep neural network (RF+DNN) classification framework. In this study, the spatial characteristics of band information, vegetation index, and polarization of main crops in the study area were constructed using Sentinel-1 and Sentinel-2 data. The temporal characteristics of crops phenology and growth state were analyzed using the curve curvature method, and the data were screened in time and space. By comparing and analyzing the accuracy of the four classification methods, the advantages of RF+DNN model and its application value in crops classification were illustrated. The results showed that for the crops in the study area during the period of good growth and development, a better crop classification result could be obtained using RF+DNN classification method, whose model accuracy, training, and predict time spent were better than that of using DNN alone. The overall accuracy and Kappa coefficient of classification were 0.98 and 0.97, respectively. It is also higher than the classification accuracy of random forest (OA = 0.87, Kappa = 0.82), object oriented (OA = 0.78, Kappa = 0.70) and deep neural network (OA = 0.93, Kappa = 0.90). The scalable and simple classification method proposed in this paper gives full play to the advantages of cloud platform in data and operation, and the traditional machine learning combined with deep learning can effectively improve the classification accuracy. Timely and accurate extraction of crop types at different spatial and temporal scales is of great significance for crops pattern change, crops yield estimation, and crops safety warning. [ABSTRACT FROM AUTHOR]

Network pruning has been widely used in model compression techniques, and offers a promising prospect for deploying models on devices with limited resources. Nevertheless, existing pruning methods merely consider the importance of feature maps and filters in the spatial domain. In this paper, we re-consider the model characteristics and propose a novel filter pruning method that corresponds to the human visual system, termed Low Frequency Preference (LFP), in the frequency domain. It is essentially an indicator that determines the importance of a filter based on the relative low-frequency components across channels, which can be intuitively understood as a measurement of the "low-frequency components". When the feature map of a filter has more low-frequency components than the other feature maps, it is considered more crucial and should be preserved during the pruning process. We conduct the proposed LFP on three different scales of datasets through several models and achieve superior performances. The experimental results obtained on the CIFAR datasets and ImageNet dataset demonstrate that our method significantly reduces the model size and FLOPs. The results on the UC Merced dataset show that our approach is also significant for remote sensing image classification. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have been widely utilized in automatic visual navigation and recognition on modern unmanned aerial vehicles (UAVs), achieving state-of-the-art performances. However, DNN-based visual recognition systems on UAVs show serious vulnerability to adversarial camouflage patterns on targets and well-designed imperceptible perturbations in real-time images, which poses a threat to safety-related applications. Considering a scenario in which a UAV is suffering from adversarial attack, in this paper, we investigate and construct two ensemble approaches with CNN and transformer for both proactive (i.e., generate robust models) and reactive (i.e., adversarial detection) adversarial defense. They are expected to be secure under attack and adapt to the resource-limited environment on UAVs. Specifically, the probability distributions of output layers from base DNN models in the ensemble are combined in the proactive defense, which mainly exploits the weak adversarial transferability between the CNN and transformer. For the reactive defense, we integrate the scoring functions of several adversarial detectors with the hidden features and average the output confidence scores from ResNets and ViTs as a second integration. To verify their effectiveness in the recognition task of remote sensing images, we conduct experiments on both optical and synthetic aperture radar (SAR) datasets. We find that the ensemble model in proactive defense performs as well as three popular counterparts, and both of the ensemble approaches can achieve much more satisfactory results than a single base model/detector, which effectively alleviates adversarial vulnerability without extra re-training. In addition, we establish a one-stop platform for conveniently evaluating adversarial robustness and performing defense on recognition models called AREP-RSIs, which is beneficial for the future research of the remote sensing field. [ABSTRACT FROM AUTHOR]

Currently, we can solve a wide range of tasks using computer vision algorithms, which reduce manual labor and enable rapid analysis of the environment. The remote sensing domain provides vast amounts of satellite data, but it also poses challenges associated with processing this data. Baseline solutions with intermediate results are available for various tasks, such as forest species classification, infrastructure recognition, and emergency situation analysis using satellite data. Despite these advances, two major issues with high-performing artificial intelligence algorithms remain in the current decade. The first issue relates to the availability of data. To train a robust algorithm, a reasonable amount of well-annotated training data is required. The second issue is the availability of satellite data, which is another concern. Even though there are a number of data providers, high-resolution and up-to-date imagery is extremely expensive. This paper aims to address these challenges by proposing an effective pipeline for building segmentation that utilizes freely available Sentinel-2 data with 10 m spatial resolution. The approach we use combines a super-resolution (SR) component with a semantic segmentation component. As a result, we simultaneously consider and analyze SR and building segmentation tasks to improve the quality of the infrastructure analysis through medium-resolution satellite data. Additionally, we collected and made available a unique dataset for the Russian Federation covering area of 1091.2 square kilometers. The dataset provides Sentinel-2 imagery adjusted to the spatial resolution of 2.5 m and is accompanied by semantic segmentation masks. The building footprints were created using OpenStreetMap data that was manually checked and verified. Several experiments were conducted for the SR task, using advanced image SR methods such as the diffusion-based SR3 model, RCAN, SRGAN, and MCGR. The MCGR network produced the best result, with a PSNR of 27.54 and SSIM of 0.79. The obtained SR images were then used to tackle the building segmentation task with different neural network models, including DeepLabV3 with different encoders, SWIN, and Twins transformers. The SWIN transformer achieved the best results, with an F1-score of 79.60. [ABSTRACT FROM AUTHOR]

One of the challenges in the field of remote sensing is how to automatically identify and classify high-resolution remote sensing images. A number of approaches have been proposed. Among them, the methods based on low-level visual features and middle-level visual features have limitations. Therefore, this paper adopts the method of deep learning to classify scenes of high-resolution remote sensing images to learn semantic information. Most of the existing methods of convolutional neural networks are based on the existing model using transfer learning, while there are relatively few articles about designing of new convolutional neural networks based on the existing high-resolution remote sensing image datasets. In this context, this paper proposes a multi-view scaling strategy, a new convolutional neural network based on residual blocks and fusing strategy of pooling layer maps, and uses optimization methods to make the convolutional neural network named RFPNet more robust. Experiments on two benchmark remote sensing image datasets have been conducted. On the UC Merced dataset, the test accuracy, precision, recall, and F1-score all exceed 93%. On the SIRI-WHU dataset, the test accuracy, precision, recall, and F1-score all exceed 91%. Compared with the existing methods, such as the most traditional methods and some deep learning methods for scene classification of high-resolution remote sensing images, the proposed method has higher accuracy and robustness. [ABSTRACT FROM AUTHOR]

Image dehazing is crucial for improving the advanced applications on remote sensing (RS) images. However, collecting paired RS images to train the deep neural networks (DNNs) is scarcely available, and the synthetic datasets may suffer from domain-shift issues. In this paper, we propose a zero-shot RS image dehazing method based on a re-degradation haze imaging model, which directly restores the haze-free image from a single hazy image. Based on layer disentanglement, we design a dehazing framework consisting of three joint sub-modules to disentangle the hazy input image into three components: the atmospheric light, the transmission map, and the recovered haze-free image. We then generate a re-degraded hazy image by mixing up the hazy input image and the recovered haze-free image. By the proposed re-degradation haze imaging model, we theoretically demonstrate that the hazy input and the re-degraded hazy image follow a similar haze imaging model. This finding helps us to train the dehazing network in a zero-shot manner. The dehazing network is optimized to generate outputs that satisfy the relationship between the hazy input image and the re-degraded hazy image in the re-degradation haze imaging model. Therefore, given a hazy RS image, the dehazing network directly infers the haze-free image by minimizing a specific loss function. Using uniform hazy datasets, non-uniform hazy datasets, and real-world hazy images, we conducted comprehensive experiments to show that our method outperforms many state-of-the-art (SOTA) methods in processing uniform or slight/moderate non-uniform RS hazy images. In addition, evaluation on a high-level vision task (RS image road extraction) further demonstrates the effectiveness and promising performance of the proposed zero-shot dehazing method. [ABSTRACT FROM AUTHOR]

Airplane detection in remote sensing images remains a challenging problem due to the complexity of backgrounds. In recent years, with the development of deep learning, object detection has also obtained great breakthroughs. For object detection tasks in natural images, such as the PASCAL (Pattern Analysis, Statistical Modelling and Computational Learning) VOC (Visual Object Classes) Challenge, the major trend of current development is to use a large amount of labeled classification data to pre-train the deep neural network as a base network, and then use a small amount of annotated detection data to fine-tune the network for detection. In this paper, we use object detection technology based on deep learning for airplane detection in remote sensing images. In addition to using some characteristics of remote sensing images, some new data augmentation techniques have been proposed. We also use transfer learning and adopt a single deep convolutional neural network and limited training samples to implement end-to-end trainable airplane detection. Classification and positioning are no longer divided into multistage tasks; end-to-end detection attempts to combine them for optimization, which ensures an optimal solution for the final stage. In our experiment, we use remote sensing images of airports collected from Google Earth. The experimental results show that the proposed algorithm is highly accurate and meaningful for remote sensing object detection. [ABSTRACT FROM AUTHOR]

Background: Road network data are crucial in various applications, such as emergency response, urban planning, and transportation management. The recent application of deep neural networks has significantly boosted the efficiency and accuracy of road network extraction based on remote sensing data. However, most existing methods for road extraction were designed at local or regional scales. Automatic extraction of large-scale road datasets from satellite images remains challenging due to the complex background around the roads, especially the complicated land cover types. To tackle this issue, this paper proposes a land cover background-adaptive framework for large-scale road extraction. Method: A large number of sample image blocks (6820) are selected from six different countries of a wide region as the dataset. OpenStreetMap (OSM) is automatically converted to the ground truth of networks, and Esri 2020 Land Cover Dataset is taken as the background land cover information. A fuzzy C-means clustering algorithm is first applied to cluster the sample images according to the proportion of certain land use types that obviously negatively affect road extraction performance. Then, the specific model is trained on the images clustered as abundant with that certain land use type, while a general model is trained based on the rest of the images. Finally, the road extraction results obtained by those general and specific modes are combined. Results: The dataset selection and algorithm implementation were conducted on the cloud-based geoinformation platform Google Earth Engine (GEE) and Google Colaboratory. Experimental results showed that the proposed framework achieved stronger adaptivity on large-scale road extraction in both visual and statistical analysis. The C-means clustering algorithm applied in this study outperformed other hard clustering algorithms. Significance: The promising potential of the proposed background-adaptive network was demonstrated in the automatic extraction of large-scale road networks from satellite images as well as other object detection tasks. This search demonstrated a new paradigm for the study of large-scale remote sensing applications based on deep neural networks. [ABSTRACT FROM AUTHOR]

Soil organic matter (SOM) is an important source of nutrients required during crop growth and is an important component of cultivated soil. In this paper, we studied the possibility of using deep learning methods to establish a multi-feature model to predict SOM content. Moreover, using Nong'an County of Changchun City as the study area, Sentinel-2A remote sensing images were taken as the data source to construct the dataset by using field sampling and image processing. The LeNet-5 convolutional neural network model was chosen as the deep learning model, which was improved based on the basic model. The evaluation metrics were selected as the root mean square error (RMSE) and the coefficient of determination R2. Through comparison, the R2 of the improved model was found to be higher than that of the linear regression method, Support Vector Machines (SVM) (RMSE = 2.471, R2 = 0.4035), and Random Forest (RF) (RMSE = 2.577, R2 = 0.4913). The result shows that: (1) It is feasible to use the multispectral data extracted from remote sensing images for soil organic matter content inversion based on the deep learning model with a minimum RMSE of 2.979 and with the R2 reaching 0.89. (2) The choice of features has an impact on the prediction of the model to a certain extent. After ranking the importance of features, selecting the appropriate number of features for inversion provides better results than full feature inversion, and the computational speed is improved. [ABSTRACT FROM AUTHOR]

Extracting water bodies is an important task in remote sensing imagery (RSI) interpretation. Deep convolution neural networks (DCNNs) show great potential in feature learning; they are widely used in the water body interpretation of RSI. However, the accuracy of DCNNs is still unsatisfactory due to differences in the many hetero-features of water bodies, such as spectrum, geometry, and spatial size. To address the problem mentioned above, this paper proposes a multiscale normalization attention network (MSNANet) which can accurately extract water bodies in complicated scenarios. First of all, a multiscale normalization attention (MSNA) module was designed to merge multiscale water body features and highlight feature representation. Then, an optimized atrous spatial pyramid pooling (OASPP) module was developed to refine the representation by leveraging context information, which improves segmentation performance. Furthermore, a head module (FEH) for feature enhancing was devised to realize high-level feature enhancement and reduce training time. The extensive experiments were carried out on two benchmarks: the Surface Water dataset and the Qinghai–Tibet Plateau Lake dataset. The results indicate that the proposed model outperforms current mainstream models on OA (overall accuracy), f1-score, kappa, and MIoU (mean intersection over union). Moreover, the effectiveness of the proposed modules was proven to be favorable through ablation study. [ABSTRACT FROM AUTHOR]

SAR-optical images from different sensors can provide consistent information for scene classification. However, the utilization of unlabeled SAR-optical images in deep learning-based remote sensing image interpretation remains an open issue. In recent years, contrastive self-supervised learning (CSSL) methods have shown great potential for obtaining meaningful feature representations from massive amounts of unlabeled data. This paper investigates the effectiveness of CSSL-based pretraining models for SAR-optical remote-sensing classification. Firstly, we analyze the contrastive strategies of single-source and multi-source SAR-optical data augmentation under different CSSL architectures. We find that the CSSL framework without explicit negative sample selection naturally fits the multi-source learning problem. Secondly, we find that the registered SAR-optical images can guide the Siamese self-supervised network without negative samples to learn shared features, which is also the reason why the CSSL framework outperforms the CSSL framework with negative samples. Finally, we apply the CSSL pretrained network without negative samples that can learn the shared features of SAR-optical images to the downstream domain adaptation task of optical transfer to SAR images. We find that the choice of a pretrained network is important for downstream tasks. [ABSTRACT FROM AUTHOR]

Rice is one of the most important food crops for human beings. The timely and accurate understanding of the distribution of rice can provide an important scientific basis for food security, agricultural policy formulation, and regional development planning. As an active remote sensing system, polarimetric synthetic aperture radar (PolSAR) has the advantage of working both day and night and in all weather conditions and hence plays an important role in rice growing area identification. This paper focuses on the topic of rice planting area identification using multi-temporal PolSAR images and a deep learning method. A rice planting area identification attention U-Net (RIAU-Net) model is proposed, which is trained by multi-temporal Sentinel-1 dual-polarimetric images acquired in different periods of rice growth. In addition, considering the diversity of the rice growth period in different years caused by the different climatic conditions and other factors, a transfer mechanism is investigated to apply the well-trained model to monitor the rice planting areas in different years. The experimental results show that the proposed method can significantly improve the classification accuracy, with 11–14% F1-score improvement compared with the traditional methods and a pleasing generalization ability in different years. Moreover, the classified rice planting regions are continuous. For reproducibility, the source codes of the well-trained RIAU-Net model are provided. [ABSTRACT FROM AUTHOR]

Biomass is a key biophysical parameter for precision agriculture and plant breeding. Fast, accurate and non-destructive monitoring of biomass enables various applications related to crop growth. In this paper, strawberry dry biomass weight was modeled using 4 canopy geometric parameters (area, average height, volume, standard deviation of height) and 25 spectral variables (5 band original reflectance values and 20 vegetation indices (VIs)) extracted from the Unmanned Aerial Vehicle (UAV) multispectral imagery. Six regression techniques—multiple linear regression (MLR), random forest (RF), support vector machine (SVM), multivariate adaptive regression splines (MARS), eXtreme Gradient Boosting (XGBoost) and artificial neural network (ANN)—were employed and evaluated for biomass prediction. The ANN had the highest accuracy in a five-fold cross-validation, with R2 of 0.89~0.93, RMSE of 7.16~8.98 g and MAE of 5.06~6.29 g. As for the other five models, the addition of VIs increased the R2 from 0.77~0.80 to 0.83~0.86, and reduced the RMSE from 8.89~9.58 to 7.35~8.09 g and the MAE from 6.30~6.70 to 5.25~5.47 g, respectively. Red-edge-related VIs, including the normalized difference red-edge index (NDRE), simple ratio vegetation index red-edge (SRRedEdge), modified simple ratio red-edge (MSRRedEdge) and chlorophyll index red and red-edge (CIred&RE), were the most influential VIs for biomass modeling. In conclusion, the combination of canopy geometric parameters and VIs obtained from the UAV imagery was effective for strawberry dry biomass estimation using machine learning models. [ABSTRACT FROM AUTHOR]

Recent years have witnessed rapid development and remarkable achievements on deep learning object detection in remote sensing (RS) images. The growing improvement of the accuracy is inseparable from the increasingly complex deep convolutional neural network and the huge amount of sample data. However, the under-fitting neural network will damage the detection performance facing the difficulty of sample acquisition. Thus, it evolves into few-shot object detection (FSOD). In this article, we first briefly introduce the object detection task and its algorithms, to better understand the basic detection frameworks followed by FSOD. Then, FSOD design methods in RS images for three important aspects, such as sample, model, and learning strategy, are respectively discussed. In addition, some valuable research results of FSOD in computer vision field are also included. We advocate a wide research technique route, and some advice about feature enhancement and multi-modal fusion, semantics extraction and cross-domain mapping, fine-tune and meta-learning strategies, and so on, are provided. Based on our stated research route, a novel few-shot detector that focuses on contextual information is proposed. At the end of the paper, we summarize accuracy performance on experimental datasets to illustrate the achievements and shortcomings of the stated algorithms, and highlight the future opportunities and challenges of FSOD in RS image interpretation, in the hope of providing insights into future research. [ABSTRACT FROM AUTHOR]

Remote sensing sensors-based image processing techniques have been widely applied in non-destructive quality inspection systems of agricultural crops. Image processing and analysis were performed with computer vision and external grading systems by general and standard steps, such as image acquisition, pre-processing and segmentation, extraction and classification of image characteristics. This paper describes the design and implementation of a real-time fresh fruit bunch (FFB) maturity classification system for palm oil based on unrestricted remote sensing (CCD camera sensor) and image processing techniques using five multivariate techniques (statistics, histograms, Gabor wavelets, GLCM and BGLAM) to extract fruit image characteristics and incorporate information on palm oil species classification FFB and maturity testing. To optimize the proposed solution in terms of performance reporting and processing time, supervised classifiers, such as support vector machine (SVM), K-nearest neighbor (KNN) and artificial neural network (ANN), were performed and evaluated via ROC and AUC measurements. The experimental results showed that the FFB classification system of non-destructive palm oil maturation in real time provided a significant result. Although the SVM classifier is generally a robust classifier, ANN has better performance due to the natural noise of the data. The highest precision was obtained on the basis of the ANN and BGLAM algorithms applied to the texture of the fruit. In particular, the robust image processing algorithm based on BGLAM feature extraction technology and the ANN classifier largely provided a high AUC test accuracy of over 93% and an image-processing time of 0,44 (s) for the detection of FFB palm oil species. [ABSTRACT FROM AUTHOR]

This paper presents the Dominant Species-Physiognomy-Ecological (DSPE) classification system developed for large-scale differentiation of plant ecological communities from high-spatial resolution remote sensing images. In this system, the plant ecological communities are defined with the inference of dominant species, physiognomy, and shared ecological settings by incorporating multiple strata. The DSPE system was implemented in a cool-temperate climate zone at a regional scale. The deep recurrent neural networks with bootstrap resampling method were employed for evaluating performance of the DSPE classification using Sentinel-2 images at 10 m spatial resolution. The performance of differentiating DSPE communities was compared with the differentiation of higher, Dominant Genus-Physiognomy-Ecological (DGPE) communities. Overall, there was a small difference in the classification between 58 DSPE communities (F1-score = 85.5%, Kappa coefficient = 84.7%) and 45 DGPE communities (F1-score = 86.5%, Kappa coefficient = 85.7%). However, the class wise accuracy analysis showed that all 58 DSPE communities were differentiated with more than 60% accuracy, whereas more than 70% accuracy was obtained for the classification of all 45 DGPE communities. Since all 58 DSPE communities were classified with more than 60% accuracy, the DSPE classification system was still effective for the differentiation of plant ecological communities from satellite images at a regional scale, indicating its applications in other regions in the world. [ABSTRACT FROM AUTHOR]

As one kind of remote sensing image (RSI), Directional Polarimetric Camera (DPC) data are of great significance in atmospheric radiation transfer and climate feedback. The availability of DPC images is often hindered by clouds, and effective cloud detection is the premise of many applications. Conventional threshold-based cloud detection methods are limited in performance and generalization capability. In this paper, we propose an effective learning-based 3D multimodal fusion cloud detection network (MFCD-Net) model. The network is a three-input stream architecture with a 3D-Unet-like encoder-decoder structure to fuse the multiple modalities of reflectance image, polarization image Q, and polarization image U in DPC imagery, with consideration of the angle and spectral information. Furthermore, cross attention is utilized in fusing the polarization features into the spatial-angle-spectral features in the reflectance image to enhance the expression of the fused features. The dataset used in this paper is obtained from the DPC cloud product and the cloud mask product. The proposed MFCD-Net achieved excellent cloud detection performance, with a recognition accuracy of 95.74%, according to the results of the experiments. [ABSTRACT FROM AUTHOR]

The rapid and accurate discovery of damage information of the affected buildings is of great significance for postdisaster emergency rescue. In some related studies, the models involved can detect damaged buildings relatively accurately, but their time cost is high. Models that can guarantee both detection accuracy and high efficiency are urgently needed. In this paper, we propose a new transfer-learning deep attention network (TDA-Net). It can achieve a balance of accuracy and efficiency. The benchmarking network for TDA-Net uses a pair of deep residual networks and is pretrained on a large-scale dataset of disaster-damaged buildings. The pretrained deep residual networks have strong sensing properties on the damage information, which ensures the effectiveness of the network in prefeature grasping. In order to make the network have a more robust perception of changing features, a set of deep attention bidirectional encoding and decoding modules is connected after the TDA-Net benchmark network. When performing a new task, only a small number of samples are needed to train the network, and the damage information of buildings in the whole area can be extracted. The bidirectional encoding and decoding structure of the network allows two images to be input into the model independently, which can effectively capture the features of a single image, thereby improving the detection accuracy. Our experiments on the xView2 dataset and three datasets of disaster regions achieve high detection accuracy, which demonstrates the feasibility of our method. [ABSTRACT FROM AUTHOR]

Completing point clouds from partial terrestrial laser scannings (TLS) is a fundamental step for many 3D visual applications, such as remote sensing, digital city and autonomous driving. However, existing methods mainly followed an ordinary auto-encoder architecture with only partial point clouds as inputs, and adopted K-Nearest Neighbors (KNN) operations to extract local geometric features, which takes insufficient advantage of input point clouds and has limited ability to extract features from long-range geometric relationships, respectively. In this paper, we propose a keypoints-aligned siamese (KASiam) network for the completion of partial TLS point clouds. The network follows a novel siamese auto-encoder architecture, to learn prior geometric information of complete shapes by aligning keypoints of complete-partial pairs during the stage of training. Moreover, we propose two essential blocks cross-attention perception (CAP) and self-attention augment (SAA), which replace KNN operations with attention mechanisms and are able to establish long-range geometric relationships among points by selecting neighborhoods adaptively at the global level. Experiments are conducted on widely used benchmarks and several TLS data, which demonstrate that our method outperforms other state-of-the-art methods by a 4.72% reduction of the average Chamfer Distance of categories in PCN dataset at least, and can generate finer shapes of point clouds on partial TLS data. [ABSTRACT FROM AUTHOR]

Remote sensing image matching is the basis upon which to obtain integrated observations and complementary information representation of the same scene from multiple source sensors, which is a prerequisite for remote sensing tasks such as remote sensing image fusion and change detection. However, the intricate geometric and radiometric differences between the multimodal images render the registration quite challenging. Although multimodal remote sensing image matching methods have been developed in recent decades, most classical and deep learning based techniques cannot effectively extract high repeatable keypoints and discriminative descriptors for multimodal images. Therefore, we propose a two-step "detection + matching" framework in this paper, where each step consists of a deep neural network. A self-supervised detection network is first designed to generate similar keypoint feature maps between multimodal images, which is used to detect highly repeatable keypoints. We then propose a cross-fusion matching network, which aims to exploit global optimization and fusion information for cross-modal feature descriptors and matching. The experiments show that the proposed method has superior feature detection and matching performance compared with current state-of-the-art methods. Specifically, the keypoint repetition rate of the detection network and the NN mAP of the matching network are 0.435 and 0.712 on test datasets, respectively. The proposed whole pipeline framework is evaluated, which achieves an average M.S. and RMSE of 0.298 and 3.41, respectively. This provides a novel solution for the joint use of multimodal remote sensing images for observation and localization. [ABSTRACT FROM AUTHOR]

A water body is a common object in remote sensing images and high-quality water body extraction is important for some further applications. With the development of deep learning (DL) in recent years, semantic segmentation technology based on deep convolution neural network (DCNN) brings a new way for automatic and high-quality body extraction from remote sensing images. Although several methods have been proposed, there exist two major problems in water body extraction, especially for high resolution remote sensing images. One is that it is difficult to effectively detect both large and small water bodies simultaneously and accurately predict the edge position of water bodies with DCNN-based methods, and the other is that DL methods need a large number of labeled samples which are often insufficient in practical application. In this paper, a novel SFnet-DA network based on the domain adaptation (DA) embedding selective self-attention (SSA) mechanism and multi-scale feature fusion (MFF) module is proposed to deal with these problems. Specially, the SSA mechanism is used to increase or decrease the space detail and semantic information, respectively, in the bottom-up branches of the network by selective feature enhancement, thus it can improve the detection capability of water bodies with drastic scale change and can prevent the prediction from being affected by other factors, such as roads and green algae. Furthermore, the MFF module is used to accurately acquire edge information by changing the number of the channel of advanced feature branches with a unique fusion method. To skip the labeling work, SFnet-DA reduces the difference in feature distribution between labeled and unlabeled datasets by building an adversarial relationship between the feature extractor and the domain classifier, so that the trained parameters of the labeled datasets can be directly used to predict the unlabeled images. Experimental results demonstrate that the proposed SFnet-DA has better performance on water body segmentation than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Remote sensing technology allows to provide information about biochemical and biophysical crop traits and monitor their spatiotemporal dynamics of agriculture ecosystems. Among multiple retrieval techniques, hybrid approaches have been found to provide outstanding accuracy, for instance, for the inference of leaf area index (LAI), fractional vegetation cover (fCover), and leaf and canopy chlorophyll content (LCC and CCC). The combination of radiative transfer models (RTMs) and data-driven models creates an advantage in the use of hybrid methods. Through this review paper, we aim to provide state-of-the-art hybrid retrieval schemes and theoretical frameworks. To achieve this, we reviewed and systematically analyzed publications over the past 22 years. We identified two hybrid-based parametric and hybrid-based nonparametric regression models and evaluated their performance for each variable of interest. From the results of our extensive literature survey, most research directions are now moving towards combining RTM and machine learning (ML) methods in a symbiotic manner. In particular, the development of ML will open up new ways to integrate innovative approaches such as integrating shallow or deep neural networks with RTM using remote sensing data to reduce errors in crop trait estimations and improve control of crop growth conditions in very large areas serving precision agriculture applications. [ABSTRACT FROM AUTHOR]

Change detection (CD) is in demand in satellite imagery processing. Inspired by the recent success of the combined transformer-CNN (convolutional neural network) model, TransCNN, originally designed for image recognition, in this paper, we present STDecoder-CD for change detection applications, which is a combination of the Siamese network ("S"), the TransCNN backbone ("T"), and three types of decoders ("Decoder"). The Type I model uses a UNet-like decoder, and the Type II decoder is defined by a combination of three modules: the difference detector, FPN (feature pyramid network), and FCN (fully convolutional network). The Type III model updates the change feature map by introducing a transformer decoder. The effectiveness and advantages of the proposed methods over the state-of-the-art alternatives were demonstrated on several CD datasets, and experimental results indicate that: (1) STDecoder-CD has excellent generalization ability and has strong robustness to pseudo-changes and noise. (2) An end-to-end CD network architecture cannot be completely free from the influence of the decoding strategy. In our case, the Type I decoder often obtained finer details than Types II and III due to its multi-scale design. (3) Using the ablation or replacing strategy to modify the three proposed decoder architectures had a limited impact on the CD performance of STDecoder-CD. To the best of our knowledge, we are the first to investigate the effect of different decoding strategies on CD tasks. [ABSTRACT FROM AUTHOR]

Featured Application: Application areas of image super-resolution: surveillance, medical diagnosis, Earth observation and remote sensing, astronomical observation, biometric information identification. The principle of image super-resolution reconstruction (SR) is to pass one or more low-resolution (LR) images through information processing technology to obtain the final high-resolution (HR) image. Convolutional neural networks (CNN) have achieved better results than traditional methods in the process of an image super-resolution reconstruction. However, if the number of neural network layers is increased blindly, it will cause a significant increase in the amount of calculation, increase the difficulty of training the network, and cause the loss of image details. Therefore, in this paper, we use a novel and effective image super-resolution reconstruction technique via fast global and local residual learning model (FGRLR). The principle is to directly train a low-resolution small image on a neural network without enlarging it. This will effectively reduce the amount of calculation. In addition, the stacked local residual block (LRB) structure is used for non-linear mapping, which can effectively overcome the problem of image degradation. After extracting features, use 1 × 1 convolution to perform dimensional compression, and expand the dimensions after non-linear mapping, which can reduce the calculation amount of the model. In the reconstruction layer, deconvolution is used to enlarge the image to the required size. This also reduces the number of parameters. We use skip connections to use low-resolution information for reconstructing high-resolution images. Experimental results show that the algorithm can effectively shorten the running time without affecting the quality of image restoration. [ABSTRACT FROM AUTHOR]

To address the problems in remote sensing image change detection such as missed detection of features at different scales and incomplete region detection, this paper proposes a high-resolution remote sensing image change detection model (Multi-scale Attention Siamese Network, MASNet) based on a Siamese network and multi-scale attention mechanism. The MASNet model took the Siamese structure of the ResNet-50 network to extract features of different simultaneous images and then applied the attention module to feature maps of different scales to generate multi-scale feature representations. Meanwhile, an improved contrastive loss function was adopted to enhance the learning of change features and improving the imbalance problem between unchanged and changed samples. Furthermore, to address the current time-consuming and laborious phenomenon of manually annotating datasets, we provided a change detection dataset from Yunnan Province in China (YNCD) that contains 1540 pairs of 256 × 256 bi-temporal images with a spatial resolution of 1 m. Then, model training and change detection applications were studied by expanding a small number of experimental area samples into the existing public datasets. The results showed that the overall accuracy of the MASNet model for change detection in the experimental area is 95.34%, precision rate is 79.78%, recall rate is 81.52%, and F1 score is 80.64%, which are better than those of six comparative models (FC-EF, FC-Siam-Diff, FC-Siam-Conc, PAN, MANet, and STANet). This verifies the effectiveness of the MASNet model as well as the feasibility of change detection by expanding existing public datasets. [ABSTRACT FROM AUTHOR]

In supervised deep learning, learning good representations for remote-sensing images (RSI) relies on manual annotations. However, in the area of remote sensing, it is hard to obtain huge amounts of labeled data. Recently, self-supervised learning shows its outstanding capability to learn representations of images, especially the methods of instance discrimination. Comparing methods of instance discrimination, clustering-based methods not only view the transformations of the same image as "positive" samples but also similar images. In this paper, we propose a new clustering-based method for representation learning. We first introduce a quantity to measure representations' discriminativeness and from which we show that even distribution requires the most discriminative representations. This provides a theoretical insight into why evenly distributing the images works well. We notice that only the even distributions that preserve representations' neighborhood relations are desirable. Therefore, we develop an algorithm that translates the outputs of a neural network to achieve the goal of evenly distributing the samples while preserving outputs' neighborhood relations. Extensive experiments have demonstrated that our method can learn representations that are as good as or better than the state of the art approaches, and that our method performs computationally efficiently and robustly on various RSI datasets. [ABSTRACT FROM AUTHOR]

Object recognition, as one of the most fundamental and challenging problems in high-resolution remote sensing image interpretation, has received increasing attention in recent years. However, most conventional object recognition pipelines aim to recognize instances with bounding boxes in a supervised learning strategy, which require intensive and manual labor for instance annotation creation. In this paper, we propose a weakly supervised learning method to alleviate this problem. The core idea of our method is to recognize multiple objects in an image using only image-level semantic labels and indicate the recognized objects with location points instead of box extent. Specifically, a deep convolutional neural network is first trained to perform semantic scene classification, of which the result is employed for the categorical determination of objects in an image. Then, by back-propagating the categorical feature from the fully connected layer to the deep convolutional layer, the categorical and spatial information of an image are combined to obtain an object discriminative localization map, which can effectively indicate the salient regions of objects. Next, a dynamic updating method of local response extremum is proposed to further determine the locations of objects in an image. Finally, extensive experiments are conducted to localize aircraft and oiltanks in remote sensing images based on different convolutional neural networks. Experimental results show that the proposed method outperforms the-state-of-the-art methods, achieving the precision, recall, and F 1 -score at 94.50%, 88.79%, and 91.56% for aircraft localization and 89.12%, 83.04%, and 85.97% for oiltank localization, respectively. We hope that our work could serve as a basic reference for remote sensing object localization via a weakly supervised strategy and provide new opportunities for further research. [ABSTRACT FROM AUTHOR]

Change detection (CD) is one of the important applications of remote sensing and plays an important role in disaster assessment, land use detection, and urban sprawl tracking. High-accuracy fully supervised methods are the main methods for CD tasks at present. However, these methods require a large amount of labeled data consisting of bi-temporal images and their change maps. Moreover, creating change maps takes a lot of labor and time. To address this limitation, a simple semi-supervised change detection method based on consistency regularization and strong augmentation is proposed in this paper. First, we construct a Siamese nested UNet with graph attention mechanism (SANet) and pre-train it with a small amount of labeled data. Then, we feed the unlabeled data into the pre-trained SANet and confidence threshold filter to obtain pseudo-labels with high confidence. At the same time, we produce distorted images by performing strong augmentation on unlabeled data. The model is trained to make the CD results of the distorted images consistent with the corresponding pseudo-label. Extensive experiments are conducted on two high-resolution remote sensing datasets. The results demonstrate that our method can effectively improve the performance of change detection under insufficient labels. Our methods can increase the IoU by more than 25% compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Change detection (CD) is a particularly important task in the field of remote sensing image processing. It is of practical importance for people when making decisions about transitional situations on the Earth's surface. The existing CD methods focus on the design of feature extraction network, ignoring the strategy fusion and attention enhancement of the extracted features, which will lead to the problems of incomplete boundary of changed area and missing detection of small targets in the final output change map. To overcome the above problems, we proposed a hierarchical attention residual nested U-Net (HARNU-Net) for remote sensing image CD. First, the backbone network is composed of a Siamese network and nested U-Net. We remold the convolution block in nested U-Net and proposed ACON-Relu residual convolution block (A-R), which reduces the missed detection rate of the backbone network in small change areas. Second, this paper proposed the adjacent feature fusion module (AFFM). Based on the adjacency fusion strategy, the module effectively integrates the details and semantic information of multi-level features, so as to realize the feature complementarity and spatial mutual enhancement between adjacent features. Finally, the hierarchical attention residual module (HARM) is proposed, which locally filters and enhances the features in a more fine-grained space to output a much better change map. Adequate experiments on three challenging benchmark public datasets, CDD, LEVIR-CD and BCDD, show that our method outperforms several other state-of-the-art methods and performs excellent in F1, IOU and visual image quality. [ABSTRACT FROM AUTHOR]