Search

Soil moisture is a crucial factor in the field of meteorology, hydrology, and agricultural sciences. In agricultural production, surface soil moisture (SSM) is crucial for crop yield estimation and drought monitoring. For SSM inversion, a synthetic aperture radar (SAR) offers a trustworthy data source. However, for agricultural fields, the use of SAR data alone to invert SSM is susceptible to the influence of vegetation cover. In this paper, based on Sentinel-1 microwave remote sensing data and Sentinel-2 optical remote sensing data, a convolution neural network optimized by sparrow search algorithm (SSA-CNN) was suggested to invert farmland SSM. The feature parameters were first extracted from pre-processed remote sensing data. Then, the correlation analysis between the extracted feature parameters and field measured SSM data was carried out, and the optimal combination of feature parameters for SSM inversion was selected as the input data of the subsequent models. To enhance the performance of the CNN, the hyper-parameters of CNN were optimized using SSA, and the SSA-CNN model was built for SSM inversion based on the obtained optimal hyper-parameter combination. Three typical machine learning approaches, including generalized regression neural network, random forest, and CNN, were used for comparison to show the efficacy of the suggested method. With an average coefficient of determination of 0.80, an average root mean square error of 2.17 vol.%, and an average mean absolute error of 1.68 vol.%, the findings demonstrated that the SSA-CNN model with the optimal feature combination had a better accuracy among the 4 models. In the end, the SSM of the study region was inverted throughout four phenological periods using the SSA-CNN model. The inversion results indicated that the suggested method performed well in local situations. [ABSTRACT FROM AUTHOR]

Synthetic Aperture Radar (SAR) imagery plays an important role in observing tropical cyclones (TCs). However, the C-band attenuation caused by rain bands and the problem of signal saturation at high wind speeds make it impossible to retrieve the fine structure of TCs effectively. In this paper, a dual-level contextual attention generative adversarial network (DeCA-GAN) is tailored for reconstructing SAR wind speeds in TCs. The DeCA-GAN follows an encoder–neck–decoder architecture, which works well for high wind speeds and the reconstruction of a large range of low-quality data. A dual-level encoder comprising a convolutional neural network and a self-attention mechanism is designed to extract the local and global features of the TC structure. After feature fusion, the neck explores the contextual features to form a reconstructed outline and up-samples the features in the decoder to obtain the reconstructed results. The proposed deep learning model has been trained and validated using the European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric model product and can be directly used to improve the data quality of SAR wind speeds. Wind speeds are reconstructed well in regions of low-quality SAR data. The root mean square error of the model output and ECMWF in these regions is halved in comparison with the existing SAR wind speed product for the test set. The results indicate that deep learning methods are effective for reconstructing SAR wind speeds. [ABSTRACT FROM AUTHOR]

Forests are critical to mitigating global climate change and regulating climate through their role in the global carbon and water cycles. Accurate monitoring of forest cover is, therefore, essential. Image segmentation networks based on convolutional neural networks have shown significant advantages in remote sensing image analysis with the development of deep learning. However, deep learning networks typically require a large amount of manual ground truth labels for training, and existing widely used image segmentation networks struggle to extract details from large-scale high resolution satellite imagery. Improving the accuracy of forest image segmentation remains a challenge. To reduce the cost of manual labelling, this paper proposed a data augmentation method that expands the training data by modifying the spatial distribution of forest remote sensing images. In addition, to improve the ability of the network to extract multi-scale detailed features and the feature information from the NIR band of satellite images, we proposed a high-resolution forest remote sensing image segmentation network by fusing multi-scale features based on double input. The experimental results using the Sanjiangyuan plateau forest dataset show that our method achieves an IoU of 90.19%, which outperforms prevalent image segmentation networks. These results demonstrate that the proposed approaches can extract forests from remote sensing images more effectively and accurately. [ABSTRACT FROM AUTHOR]

In recent years, the application of semantic segmentation methods based on the remote sensing of images has become increasingly prevalent across a diverse range of domains, including but not limited to forest detection, water body detection, urban rail transportation planning, and building extraction. With the incorporation of the Transformer model into computer vision, the efficacy and accuracy of these algorithms have been significantly enhanced. Nevertheless, the Transformer model's high computational complexity and dependence on a pre-training weight of large datasets leads to a slow convergence during the training for remote sensing segmentation tasks. Motivated by the success of the adapter module in the field of natural language processing, this paper presents a novel adapter module (ResAttn) for improving the model training speed for remote sensing segmentation. The ResAttn adopts a dual-attention structure in order to capture the interdependencies between sets of features, thereby improving its global modeling capabilities, and introduces a Swin Transformer-like down-sampling method to reduce information loss and retain the original architecture while reducing the resolution. In addition, the existing Transformer model is limited in its ability to capture local high-frequency information, which can lead to an inadequate extraction of edge and texture features. To address these issues, this paper proposes a Local Feature Extractor (LFE) module, which is based on a convolutional neural network (CNN), and incorporates multi-scale feature extraction and residual structure to effectively overcome this limitation. Further, a mask-based segmentation method is employed and a residual-enhanced deformable attention block (Deformer Block) is incorporated to improve the small target segmentation accuracy. Finally, a sufficient number of experiments were performed on the ISPRS Potsdam datasets. The experimental results demonstrate the superior performance of the model described in this paper. [ABSTRACT FROM AUTHOR]

High-resolution remote sensing image change detection technology compares and analyzes bi-temporal or multitemporal high-resolution remote sensing images to determine the change areas. It plays an important role in land cover/use monitoring, natural disaster monitoring, illegal building investigation, military target strike effect analysis, and land and resource investigation. The change detection of high-resolution remote sensing images has developed rapidly from data accumulation to algorithm models because of the rapid development of technologies such as deep learning and earth observation in recent years. However, the current deep learning-based change detection methods are strongly dependent on large sample data, and the training model has insufficient cross-domain generalization ability. As a result, a prior semantic information-guided change detection framework (PSI-CD), which alleviates the change detection model's dependence on datasets by making full use of prior semantic information, is proposed in this paper. The proposed method mainly includes two parts: one is a prior semantic information generation network that uses the semantic segmentation dataset to extract robust and reliable prior semantic information; the other is the prior semantic information guided change detection network that makes full use of prior semantic information to reduce the sample size of the change detection. To verify the effectiveness of the proposed method, we produced pixel-level semantic labels for the bi-temporal images of the public change detection dataset (LEVIR-CD). Then, we performed extensive experiments on the WHU and LEVIR-CD datasets, including comparisons with existing methods, experiments with different amounts of data, and ablation study, to show the effectiveness of the proposed method. Compared with other existing methods, our method has the highest IoU for all training samples and different amounts of training samples on WHU and LEVIR-CD, reaching a maximum of 83.25% and 83.80%, respectively. [ABSTRACT FROM AUTHOR]

Marine oil spills can cause serious damage to marine ecosystems and biological species, and the pollution is difficult to repair in the short term. Accurate oil type identification and oil thickness quantification are of great significance for marine oil spill emergency response and damage assessment. In recent years, hyperspectral remote sensing technology has become an effective means to monitor marine oil spills. The spectral and spatial features of oil spill images at different levels are different. To accurately identify oil spill types and quantify oil film thickness, and perform better extraction of spectral and spatial features, a multilevel spatial and spectral feature extraction network is proposed in this study. First, the graph convolutional neural network and graph attentional neural network models were used to extract spectral and spatial features in non-Euclidean space, respectively, and then the designed modules based on 2D expansion convolution, depth convolution, and point convolution were applied to extract feature information in Euclidean space; after that, a multilevel feature fusion method was developed to fuse the obtained spatial and spectral features in Euclidean space in a complementary way to obtain multilevel features. Finally, the multilevel features were fused at the feature level to obtain the oil spill information. The experimental results show that compared with CGCNN, SSRN, and A2S2KResNet algorithms, the accuracy of oil type identification and oil film thickness classification of the proposed method in this paper is improved by 12.82%, 0.06%, and 0.08% and 2.23%, 0.69%, and 0.47%, respectively, which proves that the method in this paper can effectively extract oil spill information and identify different oil spill types and different oil film thicknesses. [ABSTRACT FROM AUTHOR]

Autonomous navigation technology is a core technology for intelligent operation, allowing the vehicles to perform tasks without relying on external information, which effectively improves the concealability and reliability. In this paper, based on the previous research on the bionic compound eye, a multi-channel camera array with different polarization degrees was used to construct the atmospheric polarization state measurement platform. A polarization trough threshold segmentation algorithm was applied to study the distribution characteristics and characterization methods of polarization states in atmospheric remote sensing images. In the extracted polarization feature map, the tilting suggestion box was obtained based on the multi-direction window extraction network (similarity-based region proposal networks, SRPN) and the rotation of the suggestion box (Rotation Region of interests, RRoIs). Fast Region Convolutional Neural Networks (RCNN) was used to screen the suggestion boxes, and the Non-maximum suppression (NMS) method was used to select the angle, corresponding to the label of the suggestion box with the highest score, as the solar meridian azimuth in the vehicle coordinate system. The azimuth angle of the solar meridian in the atmospheric coordinate system can be calculated by the astronomical formula. Finally, the final heading angle can be obtained according to the conversion relationship between the coordinate systems. By fitting the measured data based on the least Square method, the slope K value is −1.062, RMSE (Root Mean Square Error) is 6.984, and the determination coefficient R-Square is 0.9968. Experimental results prove the effectiveness of the proposed algorithm, and this study can construct an autonomous navigation algorithm with high concealment and precision, providing a new research idea for the research of autonomous navigation technology. [ABSTRACT FROM AUTHOR]

The application of Transformer in computer vision has had the most significant influence of all the deep learning developments over the past five years. In addition to the exceptional performance of convolutional neural networks (CNN) in hyperspectral image (HSI) classification, Transformer has begun to be applied to HSI classification. However, for the time being, Transformer has not produced satisfactory results in HSI classification. Recently, in the field of image classification, the creators of Sequencer have proposed a Sequencer structure that substitutes the Transformer self-attention layer with a BiLSTM2D layer and achieves satisfactory results. As a result, this paper proposes a unique network called SquconvNet, that combines CNN with Sequencer block to improve hyperspectral classification. In this paper, we conducted rigorous HSI classification experiments on three relevant baseline datasets to evaluate the performance of the proposed method. The experimental results show that our proposed method has clear advantages in terms of classification accuracy and stability. [ABSTRACT FROM AUTHOR]

Clouds seriously limit the application of optical remote sensing images. In this paper, we remove clouds from satellite images using a novel method that considers ground surface reflections and cloud top reflections as a linear mixture of image elements from the perspective of image superposition. We use a two-step convolutional neural network to extract the transparency information of clouds and then recover the ground surface information of thin cloud regions. Given the poor balance of the generated samples, this paper also improves the binary Tversky loss function and applies it on multi-classification tasks. The model was validated on the simulated dataset and ALCD dataset, respectively. The results show that this model outperformed other control group experiments in cloud detection and removal. The model better locates the clouds in images with cloud matting, which is built based on cloud detection. In addition, the model successfully recovers the surface information of the thin cloud region when thick and thin clouds coexist, and it does not damage the original image's information. [ABSTRACT FROM AUTHOR]

Due to the imaging mechanism of synthetic aperture radar (SAR), it is difficult and costly to acquire abundant labeled SAR images. Moreover, a typical matched filtering (MF) based image faces the problems of serious noise, sidelobes, and clutters, which will bring down the accuracy of SAR target classification. Different from the MF-based result, a sparse image shows better quality with less noise and higher image signal-to-noise ratio (SNR). Therefore, theoretically using it for target classification will achieve better performance. In this paper, a novel contrastive domain adaptation (CDA) based sparse SAR target classification method is proposed to solve the problem of insufficient samples. In the proposed method, we firstly construct a sparse SAR image dataset by using the complex image based iterative soft thresholding (BiIST) algorithm. Then, the simulated and real SAR datasets are simultaneously sent into an unsupervised domain adaptation framework to reduce the distribution difference and obtain the reconstructed simulated SAR images for subsequent target classification. Finally, the reconstructed simulated images are manually labeled and fed into a shallow convolutional neural network (CNN) for target classification along with a small number of real sparse SAR images. Since the current definition of the number of small samples is still vague and inconsistent, this paper defines few-shot as less than 20 per class. Experimental results based on MSTAR under standard operating conditions (SOC) and extended operating conditions (EOC) show that the reconstructed simulated SAR dataset makes up for the insufficient information from limited real data. Compared with other typical deep learning methods based on limited samples, our method is able to achieve higher accuracy especially under the conditions of few shots. [ABSTRACT FROM AUTHOR]

The Earth rotation parameters (ERPs), including polar motion (PMX and PMY) and universal time (UT1-UTC), play a central role in functions such as monitoring the Earth's rotation and high-precision navigation and positioning. Variations in ERPs reflect not only the overall state of movement of the Earth, but also the interactions among the atmosphere, ocean, and land on the spatial and temporal scales. In this paper, we estimated ERP series based on very long baseline interferometry (VLBI) observations between 2011–2020. The results show that the average root mean square errors (RMSEs) are 0.187 mas for PMX, 0.205 mas for PMY, and 0.022 ms for UT1-UTC. Furthermore, to explore the high-frequency variations in more detail, we analyzed the polar motion time series spectrum based on fast Fourier transform (FFT), and our findings show that the Chandler motion was approximately 426 days and that the annual motion was about 360 days. In addition, the results also validate the presence of a weaker retrograde oscillation with an amplitude of about 3.5 mas. This paper proposes a hybrid prediction model that combines convolutional neural network (CNN) and long short-term memory (LSTM) neural network: the CNN–LSTM model. The advantages can be attributed to the CNN's ability to extract and optimize features related to polar motion series, and the LSTM's ability to make medium- to long-term predictions based on historical time series. Compared with Bulletin A, the prediction accuracies of PMX and PMY are improved by 42% and 13%, respectively. Notably, the hybrid CNN–LSTM model can effectively improve the accuracy of medium- and long-term polar motion prediction. [ABSTRACT FROM AUTHOR]

To meet the demands of natural resource monitoring, land development supervision, and other applications for high-precision and high-frequency information extraction from constructed land change, this paper focused on automatic feature extraction and data processing optimization methods for newly constructed bare land based on remote sensing images. A generalized deep convolutional neural network change detection model framework integrating multi-scale information was developed for the automatic extraction of change information. To resolve the problems in the automatic extraction of new bare land parcels, such as mis-extractions and parcel fragmentation, a proximity evaluation model that integrates the confidence-based semantic distance and spatial distance between parcels and their overlapping area is proposed to perform parcel aggregation. Additionally, we propose a complete set of optimized processing techniques from pixel pre-processing to vector post-processing. The results demonstrated that the aggregation method developed in this study is more targeted and effective than ArcGIS for the automatically extracted land change parcels. Additionally, compared with the initial parcels, the total number of optimized parcels decreased by more than 50% and the false detection rate decreased by approximately 30%. These results indicate that this method can markedly reduce the overall data volume and false detection rate of automatically extracted parcels through post-processing under certain conditions of the model and samples and provide technical support for applying the results of automatic feature extraction in engineering practices. [ABSTRACT FROM AUTHOR]

Convolution-based recurrent neural networks and convolutional neural networks have been used extensively in spatiotemporal prediction. However, these methods tend to concentrate on fixed-scale spatiotemporal state transitions and disregard the complexity of spatiotemporal motion. Through statistical analysis, we found that the distribution of the spatiotemporal sequence and the variety of spatiotemporal motion state transitions exhibit some regularity. In light of these statistics and observations, we propose the Multi-scale Spatiotemporal Neural Network (MSSTNet), an end-to-end neural network based on 3D convolution. It can be separated into three major child modules: a distribution feature extraction module, a multi-scale motion state capture module, and a feature decoding module. Furthermore, the MSST unit is designed to model multi-scale spatial and temporal information in the multi-scale motion state capture module. We first conduct the experiments on the MovingMNIST dataset, which is the most commonly used dataset in the field of spatiotemporal prediction, MSSTNet can achieve state-of-the-art results for this dataset, and ablation experiments demonstrate that the MSST unit has positive significance for spatiotemporal prediction. In addition, this paper applies the model to valuable precipitation nowcasting, due to efficiently capturing the multi-scale information of distribution and motion, the new MSSTNet model can predict the real-world radar echo more accurately. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural networks (CNNs) have gained widespread adoption in remote sensing image processing. Deploying CNN-based algorithms on satellite edge devices can alleviate the strain on data downlinks. However, CNN algorithms present challenges due to their large parameter count and high computational requirements, which conflict with the satellite platforms' low power consumption and high real-time requirements. Moreover, remote sensing image processing tasks are diverse, requiring the platform to accommodate various network structures. To address these issues, this paper proposes an algorithm–hardware co-optimization and deployment method for FPGA-based CNN remote sensing image processing. Firstly, a series of hardware-centric model optimization techniques are proposed, including operator fusion and depth-first mapping technology, to minimize the resource overhead of CNN models. Furthermore, a versatile hardware accelerator is proposed to accelerate a wide range of commonly used CNN models after optimization. The accelerator architecture mainly consists of a parallel configurable network processing unit and a multi-level storage structure, enabling the processing of optimized networks with high throughput and low power consumption. To verify the superiority of our method, the introduced accelerator was deployed on an AMD-Xilinx VC709 evaluation board, on which the improved YOLOv2, VGG-16, and ResNet-34 networks were deployed. Experiments show that the power consumption of the accelerator is 14.97 W, and the throughput of the three networks reaches 386.74 giga operations per second (GOPS), 344.44 GOPS, and 182.34 GOPS, respectively. Comparison with related work demonstrates that the co-optimization and deployment method can accelerate remote sensing image processing CNN models and is suitable for applications in satellite edge devices. [ABSTRACT FROM AUTHOR]

Inspired by the success of Convolutional Neural Network (CNN)-based deep learning methods for optical image segmentation, there is a growing interest in applying these methods to Polarimetric Synthetic Aperture Radar (PolSAR) data. However, effectively utilizing well-established real-valued CNNs for PolSAR image segmentation requires converting complex-valued data into real-valued representations. This paper presents a systematic comparison of 14 different real-valued representations used as CNN input in the literature. These representations encompass various approaches, including the use of coherency matrix elements, hand-crafted feature vectors, polarimetric features based on target decomposition, and combinations of these methods. The goal is to assess the impact of the choice of PolSAR data representation on segmentation performance and identify the most suitable representation. Four test configurations are employed to achieve this, involving different CNN architectures (U-Net with ResNet-18 or EfficientNet backbone) and PolSAR data acquired in different frequency bands (S- and L-band). The results emphasize the importance of selecting an appropriate real-valued representation for CNN-based PolSAR image segmentation. This study's findings reveal that combining multiple polarimetric features can potentially enhance segmentation performance but does not consistently improve the results. Therefore, when employing this approach, careful feature selection becomes crucial. In contrast, using coherency matrix elements with amplitude and phase representation consistently achieves high segmentation performance across different test configurations. This representation emerges as one of the most suitable approaches for CNN-based PolSAR image segmentation. Notably, it outperforms the commonly used alternative approach of splitting the coherency matrix elements into real and imaginary parts. [ABSTRACT FROM AUTHOR]

Enhancing the spatial resolution of real-aperture microwave radiometers is an essential research topic. The accuracy of the numerical values of brightness temperatures (BTs) observed using microwave radiometers directly affects the precision of the retrieval of marine environmental parameters. Hence, ensuring the accuracy of the enhanced brightness temperature values is of paramount importance when striving to enhance spatial resolution. A spectrum extension (SE) method is proposed in this paper, which restores the suppressed high-frequency components in the scene BT spectrum through frequency domain transformation and calculations, specifically, dividing the observed BT spectrum by the conjugate of the antenna pattern spectrum and applying a Taylor approximation to suppress error amplification, thereby extending the observed BT spectrum. By using a convolutional neural network to correct errors in the calculated spectrum and then reconstructing the BT through inverse fast Fourier transform (IFFT), the enhanced BTs are obtained. Since the extended BT spectrum contains more high-frequency components, namely, the spectrum is closer to that of the original scene BT, the reconstructed BT not only achieves an enhancement in spatial resolution, but also an improvement in the accuracy of BT values. Both the results from simulated data and satellite-measured data processing illustrate that the SE method is able to enhance the spatial resolution of real-aperture microwave radiometers and concurrently improve the accuracy of BT values. [ABSTRACT FROM AUTHOR]

This paper presents the MSSFF (multistage spectral–spatial feature fusion) framework, which introduces a novel approach for semantic segmentation from hyperspectral imagery (HSI). The framework aims to simplify the modeling of spectral relationships in HSI sequences and unify the architecture for semantic segmentation of HSIs. It incorporates a spectral–spatial feature fusion module and a multi-attention mechanism to efficiently extract hyperspectral features. The MSSFF framework reevaluates the potential impact of spectral and spatial features on segmentation models and leverages the spectral–spatial fusion module (SSFM) in the encoder component to effectively extract and enhance these features. Additionally, an efficient Transformer (ET) is introduced in the skip connection part of deep features to capture long-term dependent features and extract global spectral–spatial information from the entire feature map. This highlights the significant potential of Transformers in modeling spectral–spatial feature maps within the context of hyperspectral remote sensing. Moreover, a spatial attention mechanism is adopted in the shallow skip connection part to extract local features. The framework demonstrates promising capabilities in hyperspectral remote sensing applications. The conducted experiments provide valuable insights for optimizing the model depth and the order of feature fusion, thereby contributing to the advancement of hyperspectral semantic segmentation research. [ABSTRACT FROM AUTHOR]

Remote sensing (RS) scene classification has always attracted much attention as an elemental and hot topic in the RS community. In recent years, many methods using convolutional neural networks (CNNs) and other advanced machine-learning techniques have been proposed. Their performance is excellent; however, they are disabled when there are noisy labels (i.e., RS scenes with incorrect labels), which is inevitable and common in practice. To address this problem, some specific RS classification models have been developed. Although feasible, their behavior is still limited by the complex contents of RS scenes, excessive noise filtering schemes, and intricate noise-tolerant learning strategies. To further enhance the RS classification results under the noisy scenario and overcome the above limitations, in this paper we propose a multiscale information exploration network (MIEN) and a progressive learning algorithm (PLA). MIEN involves two identical sub-networks whose goals are completing the classification and recognizing possible noisy RS scenes. In addition, we develop a transformer-assistive multiscale fusion module (TAMSFM) to enhance MIEN's behavior in exploring the local, global, and multiscale contents within RS scenes. PLA encompasses a dual-view negative-learning (DNL) stage, an adaptively positive-learning (APL) stage, and an exhaustive soft-label-learning (ESL) stage. Their aim is to learn the relationships between RS scenes and irrelevant semantics, model the links between clean RS scenes and their labels, and generate reliable pseudo-labels. This way, MIEN can be thoroughly trained under the noisy scenario. We simulate noisy scenarios and conduct extensive experiments using three public RS scene data sets. The positive experimental results demonstrate that our MIEN and PLA can fully understand RS scenes and resist the negative influence of noisy samples. [ABSTRACT FROM AUTHOR]

Spectral unmixing poses a significant challenge within hyperspectral image processing, traditionally addressed by supervised convolutional neural network (CNN)-based approaches employing patch-to-pixel (pixel-wise) methods. However, such pixel-wise methodologies often necessitate image splitting into overlapping patches, resulting in redundant computations and potential information leakage between training and test samples, consequently yielding overoptimistic outcomes. To overcome these challenges, this paper introduces a novel patch-to-patch (patch-wise) framework with nonoverlapping splitting, mitigating the need for repetitive calculations and preventing information leakage. The proposed framework incorporates a novel neural network structure inspired by the fully convolutional network (FCN), tailored for patch-wise unmixing. A highly efficient band reduction layer is incorporated to reduce the spectral dimension, and a specialized abundance constraint module is crafted to enforce both the Abundance Nonnegativity Constraint and the Abundance Sum-to-One Constraint for unmixing tasks. Furthermore, to enhance the performance of abundance estimation, a spatial–spectral attention module is introduced to activate the most informative spatial areas and feature maps. Extensive quantitative experiments and visual assessments conducted on two synthetic datasets and three real datasets substantiate the superior performance of the proposed algorithm. Significantly, the method achieves an impressive R M S E loss of 0.007, which is at least 4.5 times lower than that of other baselines on Urban hyperspectral images. This outcome demonstrates the effectiveness of our approach in addressing the challenges of spectral unmixing. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of covariance matrix estimation for radar adaptive detection under non-Gaussian clutter. Traditional model-based estimators may suffer from performance loss due to the mismatch between real data and assumed models. Therefore, we resort to a data-driven deep-learning method and propose a covariance matrix estimation method based on a complex-valued convolutional neural network (CV-CNN). Moreover, a real-valued (RV) network with the same framework as the proposed CV network is also constructed to serve as a natural competitor. The obtained clutter covariance matrix estimation based on the network is applied to the adaptive normalized matched filter (ANMF) detector for performance assessment. The detection results via both simulated and real sea clutter illustrate that the estimator based on CV-CNN outperforms other traditional model-based estimators as well as its RV competitor in terms of probability of detection (PD). [ABSTRACT FROM AUTHOR]

Geological hazards often occur in mountainous areas and are sudden and hidden, so it is important to identify and assess geological hazards. In this paper, the western mountainous area of Beijing was selected as the study area. We conducted research on landslides, collapses, and unstable slopes in the study area. The surface deformation of the study area was monitored by multi-temporal interferometric synthetic aperture radar (MT-InSAR), using a combination of multi-looking point selection and permanent scatterer (PS) point selection methods. Random forest (RF), support vector machine (SVM), convolutional neural network (CNN), and recurrent neural network (RNN) models were selected for the assessment of geological hazard susceptibility. Sixteen geological hazard-influencing factors were collected, and their information values were calculated using their features. Multicollinearity analysis with the relief-F method was used to calculate the correlation and importance of the factors for factor selection. The results show that the deformation rate along the line-of-sight (LOS) direction is between −44 mm/year and 28 mm/year. A total of 60 geological hazards were identified by combining surface deformation with optical imagery and other data, including 7 collapses, 25 unstable slopes, and 28 landslides. Forty-eight of the identified geological hazards are not recorded in the Beijing geological hazards list. The most effective model in the study area was RF. The percentage of geological hazard susceptibility zoning in the study area is as follows: very low susceptibility 27.40%, low susceptibility 28.06%, moderate susceptibility 21.19%, high susceptibility 13.80%, very high susceptibility 9.57%. [ABSTRACT FROM AUTHOR]

In recent years, significant strides have been made in the field of synthetic aperture radar (SAR) ship detection through the application of deep learning techniques. These advanced methods have substantially improved the accuracy of ship detection. Nonetheless, SAR images present distinct challenges, including complex backgrounds, small ship targets, and noise interference, thereby rendering the detectors particularly demanding. In this paper, we introduce LRTransDet, a real-time SAR ship detector. LRTransDet leverages a lightweight vision transformer (ViT) and a multi-scale feature fusion neck to address these challenges effectively. First, our model implements a lightweight backbone that combines convolutional neural networks (CNNs) and transformers, thus enabling it to simultaneously capture both local and global features from input SAR images. Moreover, we boost the model's efficiency by incorporating the faster weighted feature fusion (Faster-WF2) module and coordinate attention (CA) mechanism within the feature fusion neck. These components optimize computational resources while maintaining the model's performance. To overcome the challenge of detecting small ship targets in SAR images, we refine the original loss function and use the normalized Wasserstein distance (NWD) metric and the intersection over union (IoU) scheme. This combination improves the detector's ability to efficiently detect small targets. To prove the performance of our proposed model, we conducted experiments on four challenging datasets (the SSDD, the SAR-Ship Dataset, the HRSID, and the LS-SSDD-v1.0). The results demonstrate that our model surpasses both general object detectors and state-of-the-art SAR ship detectors in terms of detection accuracy (97.8% on the SSDD and 93.9% on the HRSID) and speed (74.6 FPS on the SSDD and 75.8 FPS on the HRSID), all while demanding 3.07 M parameters. Additionally, we conducted a series of ablation experiments to illustrate the impact of the EfficientViT, the Faster-WF2 module, the CA mechanism, and the NWD metric on multi-scale feature fusion and detection performance. [ABSTRACT FROM AUTHOR]

Estimating the complete 3D points of crop plants from incomplete points is vital for phenotyping and smart agriculture management. Compared with the completion of regular man-made objects such as airplanes, chairs, and desks, the completion of corn plant points is more difficult for thin, curled, and irregular corn leaves. This study focuses on MSGRNet+OA, which is based on GRNet, to complete a 3D point cloud of thin corn plants. The developed MSGRNet+OA was accompanied by gridding, multi-scale 3DCNN, gridding reverse, cubic feature sampling, and offset-attention. In this paper, we propose the introduction of a 3D grid as an intermediate representation to regularize the unorganized point cloud, use multi-scale predictive fusion to utilize global information at different scales, and model the geometric features by adding offset-attention to compute the point position offsets. These techniques enable the network to exhibit good adaptability and robustness in dealing with irregular and varying point cloud structures. The accuracy assessment results show that the accuracy of completion using MSGRNet+OA is superlative, with a CD (×10−4) of 1.258 and an F-Score@1% of 0.843. MSGRNet+OA is the most effective when compared with other networks (PCN, shape inversion, the original GRNet, SeedFormer, and PMP-Net++), and it improves the accuracy of the CD (×10−4)/F-Score@1% with −15.882/0.404, −15.96/0.450, −0.181/0.018, −1.852/0.274, and −1.471/0.203, respectively. These results reveal that the developed MSGRNet+OA can be used to complete a 3D point cloud of thin corn leaves for phenotyping. [ABSTRACT FROM AUTHOR]

In the complex battlefield electromagnetic environment, multiple jamming signals can enter the radar receiver simultaneously due to the development of jammers and modulation technology. The received compound jamming signals aggravate the difficulty of recognition and subsequent counter-countermeasure. In the face of strong overlapping signals and unseen jamming signal combinations, the performance of existing recognition methods usually seriously degrades. In this paper, an end-to-end multi-label classification framework combining a complex-valued convolutional neural network (CV-CNN) and jamming class representations is proposed to automatically recognize the jamming signal components of compound jamming signals. A basic multi-label CV-CNN (ML-CV-CNN) is first designed to directly process time–domain complex signals and fully retain jamming signal information. Then, the jamming class representations are generated using prototype clustering implemented by learning vector quantization, and they are fused with the ML-CV-CNN using class decoupling implemented by the attention mechanism to construct a multi-label class representation CV-CNN (ML-CR-CV-CNN), which can better learn the class-related features required for recognition. Finally, an adaptive threshold calibration is adopted to obtain optimal recognition results by multi-threshold discrimination. Simulation results verify that the proposed method has superior recognition performance, which is reflected in the strong robustness to the varying jamming-to-noise ratio (JNR) and power ratio, faster convergence speed with high JNRs, and better generalization for unseen jamming signal combinations. [ABSTRACT FROM AUTHOR]

With the continuous advancement of target detection technology in remote sensing, target detection technology in images captured by drones has performed well. However, object detection in drone imagery is still a challenge under rainy conditions. Rain is a common severe weather condition, and rain streaks often degrade the image quality of sensors. The main issue of rain streaks removal from a single image is to prevent over smoothing (or underclearing) phenomena. Aiming at the above problems, this paper proposes a deep learning (DL)-based rain streaks removal framework called GSDerainNet, which properly formulates the single image rain streaks removal problem; rain streaks removal is aptly described as a Gaussian Shannon (GS) filter-based image decomposition problem. The GS filter is a novel filter proposed by us, which consists of a parameterized Gaussian function and a scaled Shannon function. Two-dimensional GS filters exhibit high stability and effectiveness in dividing an image into low- and high-frequency parts. In our framework, an input image is first decomposed into a low-frequency part and a high-frequency part by using the GS filter. Rain streaks are located in the high-frequency part. We extract and separate the rain features of the high-frequency part through a deep convolutional neural network (CNN). The experimental results obtained on synthetic data and real data show that the proposed method can better suppress the morphological artifacts caused by filtering. Compared with state-of-the-art single image rain streaks removal methods, the proposed method retains finer image object structures while removing rain streaks. [ABSTRACT FROM AUTHOR]

Remote sensing image scene classification with deep learning (DL) is a rapidly growing field that has gained significant attention in the past few years. While previous review papers in this domain have been confined to 2020, an up-to-date review to show the progression of research extending into the present phase is lacking. In this review, we explore the recent articles, providing a thorough classification of approaches into three main categories: Convolutional Neural Network (CNN)-based, Vision Transformer (ViT)-based, and Generative Adversarial Network (GAN)-based architectures. Notably, within the CNN-based category, we further refine the classification based on specific methodologies and techniques employed. In addition, a novel and rigorous meta-analysis is performed to synthesize and analyze the findings from 50 peer-reviewed journal articles to provide valuable insights in this domain, surpassing the scope of existing review articles. Our meta-analysis shows that the most adopted remote sensing scene datasets are AID (41 articles) and NWPU-RESISC45 (40). A notable paradigm shift is seen towards the use of transformer-based models (6) starting from 2021. Furthermore, we critically discuss the findings from the review and meta-analysis, identifying challenges and future opportunities for improvement in this domain. Our up-to-date study serves as an invaluable resource for researchers seeking to contribute to this growing area of research. [ABSTRACT FROM AUTHOR]

Polarimetric synthetic aperture radar (PolSAR) image classification has been an important area of research due to its wide range of applications. Traditional machine learning methods were insufficient in achieving satisfactory results before the advent of deep learning. Results have significantly improved with the widespread use of deep learning in PolSAR image classification. However, the challenge of reconciling the complex-valued inputs of PolSAR images with the real-valued models of deep learning remains unsolved. Current complex-valued deep learning models treat complex numbers as two distinct real numbers, providing limited assistance in PolSAR image classification results. This paper proposes a novel, complex-valued deep learning approach for PolSAR image classification to address this issue. The approach includes amplitude-based max pooling, complex-valued nonlinear activation, and a cross-entropy loss function based on complex-valued probability. Amplitude-based max pooling reduces computational effort while preserving the most valuable complex-valued features. Complex-valued nonlinear activation maps feature into a high-dimensional complex-domain space, producing the most discriminative features. The complex-valued cross-entropy loss function computes the classification loss using the complex-valued model output and dataset labels, resulting in more accurate and robust classification results. The proposed method was applied to a shallow CNN, deep CNN, FCN, and SegNet, and its effectiveness was verified on three public datasets. The results showed that the method achieved optimal classification results on any model and dataset. [ABSTRACT FROM AUTHOR]

The deep learning technique has already been successfully applied in the field of microwave remote sensing. Especially, convolutional neural networks have demonstrated remarkable effectiveness in synthetic aperture radar (SAR) image semantic segmentation. In this paper, a Lovász-softmax loss optimization SAR net (LoSARNet) is proposed which optimizes the semantic segmentation metric intersection over union (IOU) instead of using the traditional cross-entropy loss. Meanwhile, making use of the advantages of the dual-path structure, the network extracts feature through the spatial path (SP) and the context path (CP) to achieve a balance between efficiency and accuracy. Aiming at a polarimetric SAR (PolSAR) image, the proposed network is conducted on the PolSAR datasets for terrain segmentation. Compared to the typical dual-path network, which is the bilateral segmentation network (BiSeNet), the proposed LoSARNet can obtain better mean intersection over union (MIOU). And the proposed network also shows the highest evaluation index and the best performance when compared with several typical networks. [ABSTRACT FROM AUTHOR]

The complex remote sensing image acquisition conditions and the differences in crop growth create many crop classification challenges. Frequency decomposition enables the capture of the feature information in an image that is difficult to discern. Frequency domain filters can strengthen or weaken specific frequency components to enhance the interclass differences among the different crops and can reduce the intraclass variations within the same crops, thereby improving crop classification accuracy. In concurrence with the Fourier frequency domain learning strategy, we propose a convolutional neural network called the Fourier frequency domain convolutional (FFDC) net, which transforms feature maps from the spatial domain to the frequency spectral domain. In this network, the dynamic frequency filtering components in the frequency spectral domain are used to separate the feature maps into low-frequency and high-frequency components, and the strength and distribution of the different frequency components are automatically adjusted to suppress the low-frequency information variations within the same crop, enhancing the overall consistency of the crops. Simultaneously, it is also used to strengthen the high-frequency information differences among the different crops to widen the interclass differences and to achieve high-precision remote sensing crop classification. In the test areas, which are randomly selected in multiple farms located far from the sampling area, we compare our method with other methods. The results demonstrate that the frequency-domain learning approach better mitigates issues, such as incomplete crop extractions and fragmented boundaries, which leads to higher classification accuracy and robustness. This paper applies frequency-domain deep learning to remote sensing crop classification, highlighting a novel and effective solution that supports agricultural management decisions and planning. [ABSTRACT FROM AUTHOR]

The recently proposed spacecraft three-dimensional (3D) structure recovery method based on optical images and LIDAR has enhanced the working distance of a spacecraft's 3D perception system. However, the existing methods ignore the richness of temporal features and fail to capture the temporal coherence of consecutive frames. This paper proposes a sequential spacecraft depth completion network (S2DCNet) for generating accurate and temporally consistent depth prediction results, and it can fully exploit temporal–spatial coherence in sequential frames. Specifically, two parallel convolution neural network (CNN) branches were first adopted to extract the features latent in different inputs. The gray image features and the depth features were hierarchically encapsulated into unified feature representations through fusion modules. In the decoding stage, the convolutional long short-term memory (ConvLSTM) networks were embedded with the multi-scale scheme to capture the feature spatial–temporal distribution variation, which could reflect the past state and generate more accurate and temporally consistent depth maps. In addition, a large-scale dataset was constructed, and the experiments revealed the outstanding performance of the proposed S2DCNet, achieving a mean absolute error of 0.192 m within the region of interest. [ABSTRACT FROM AUTHOR]

This paper presents a deep framework EddyDet to automatically detect oceanic eddies in Synthetic Aperture Radar (SAR) images. The EddyDet has been developed using the Mask Region with Convolutional Neural Networks (Mask RCNN) framework, incorporating two new branches: Edge Head and Mask Intersection over Union (IoU) Head. The Edge Head can learn internal texture information implicitly, and the Mask IoU Head improves the quality of predicted masks. A SAR dataset for Oceanic Eddy Detection (SOED) is specifically constructed to evaluate the effectiveness of the EddyDet model in detecting oceanic eddies. We demonstrate that the EddyDet is capable of achieving acceptable eddy detection results under the condition of limited training samples, which outperforms a Mask RCNN baseline in terms of average precision. The combined Edge Head and Mask IoU Head have the ability to describe the characteristics of eddies more correctly, while the EddyDet shows great potential in practice use accurately and time efficiently, saving manual labor to a large extent. [ABSTRACT FROM AUTHOR]

Road extraction is crucial in urban planning, rescue operations, and military applications. Compared to traditional methods, using deep learning for road extraction from remote sensing images has demonstrated unique advantages. However, previous convolutional neural networks (CNN)-based road extraction methods have had limited receptivity and failed to effectively capture long-distance road features. On the other hand, transformer-based methods have good global information-capturing capabilities, but face challenges in extracting road edge information. Additionally, existing excellent road extraction methods lack validation for the Chinese region. To address these issues, this paper proposes a novel road extraction model called the global–local information fusion network (GLNet). In this model, the global information extraction (GIE) module effectively integrates global contextual relationships, the local information extraction (LIE) module accurately captures road edge information, and the information fusion (IF) module combines the output features from both global and local branches to generate the final extraction results. Further, a series of experiments on two different Chinese road datasets with geographic robustness demonstrate that our model outperforms the state-of-the-art deep learning models for road extraction tasks in China. On the CHN6-CUG dataset, the overall accuracy (OA) and intersection over union (IoU) reach 97.49% and 63.27%, respectively, while on the RDCME dataset, OA and IoU reach 98.73% and 84.97%, respectively. These research results hold significant implications for road traffic, humanitarian rescue, and environmental monitoring, particularly in the context of the Chinese region. [ABSTRACT FROM AUTHOR]

The influence of some seemingly anomalous samples on modeling is often ignored in the quantitative prediction of soil composition modeling with hyperspectral data. Soil spectral transformation based on wavelet packet technology only performs pruning and threshold filtering based on experience. The feature bands selected by the Pearson correlation coefficient method often have high redundancy. To solve these problems, this paper carried out a study of the prediction of soil total iron composition based on a new method. First, regarding the problem of abnormal samples, the Monte Carlo method based on particle swarm optimization (PSO) is used to screen abnormal samples. Second, feature representation based on Shannon entropy is adopted for wavelet packet processing. The amount of information held by the wavelet packet node is used to decide whether to cut the node. Third, the feature bands selected based on the correlation coefficient and the competitive adaptive reweighted sampling (CARS) algorithm using the least squares support vector regression (LSSVR) are applied to the soil spectra before and after wavelet packet processing. Finally, the Fe content was calculated based on a 1D convolutional neural network (1D-CNN). The results show that: (1) The Monte Carlo method based on particle swarm optimization and modeling multiple times was able to handle the abnormal samples. (2) Based on the Shannon entropy wavelet packet transformation, simple operations could simultaneously preserve the spectral information while removing high-frequency noise from the spectrum, effectively improving the correlation between soil spectra and content. (3) The 1D-CNN with added residual blocks could also achieve better results in soil hyperspectral modeling with few samples. [ABSTRACT FROM AUTHOR]

In this paper, a feedback training approach for efficiently dealing with distribution shift in synthetic aperture radar target detection using a Bayesian convolutional neural network is proposed. After training the network on in-distribution data, it is tested on out-of-distribution data. Samples that are classified incorrectly with high certainty are fed back for a second round of training. This results in the reduction of false positives in the out-of-distribution dataset. False positive target detections challenge human attention, sensor resource management, and mission engagement. In these types of applications, a reduction in false positives thus often takes precedence over target detection and classification performance. The classifier is used to discriminate the targets from the clutter and to classify the target type in a single step as opposed to the traditional approach of having a sequential chain of functions for target detection and localisation before the machine learning algorithm. Another aspect of automated synthetic aperture radar detection and recognition problems addressed here is the fact that human users of the output of traditional classification systems are presented with decisions made by "black box" algorithms. Consequently, the decisions are not explainable, even to an expert in the sensor domain. This paper makes use of the concept of explainable artificial intelligence via uncertainty heat maps that are overlaid onto synthetic aperture radar imagery to furnish the user with additional information about classification decisions. These uncertainty heat maps facilitate trust in the machine learning algorithm and are derived from the uncertainty estimates of the classifications from the Bayesian convolutional neural network. These uncertainty overlays further enhance the users' ability to interpret the reasons why certain decisions were made by the algorithm. Further, it is demonstrated that feeding back the high-certainty, incorrectly classified out-of-distribution data results in an average improvement in detection performance and a reduction in uncertainty for all synthetic aperture radar images processed. Compared to the baseline method, an improvement in recall of 11.8%, and a reduction in the false positive rate of 7.08% were demonstrated using the Feedback-assisted Bayesian Convolutional Neural Network or FaBCNN. [ABSTRACT FROM AUTHOR]

The research of Synthetic aperture radar (SAR) target recognition plays a significant role in military and civilian fields. However, for small sample SAR target recognition, there are some problems that need to be solved urgently, including low recognition accuracy, slow training convergence rate, and serious overfitting. Aiming at the above problems, we propose a recognition method based on Inception and Fully Convolutional Neural Network (IFCNN) combined with Amplitude Domain Multiplicative Filtering (ADMF) image processing. To improve the recognition accuracy and convergence rate, the ADMF method is utilized to construct the pretraining set, and the initial parameters of the network are optimized by pretraining. In addition, this paper builds the IFCNN model by introducing the Inception structure and the mixed progressive convolution layer into the FCNN. The full convolution structure of FCNN is effective to alleviate the problem of network overfitting. The Inception structure can enhance the sparsity of features and improve the network classification ability. Meanwhile, the mixed progressive convolution layers can accelerate training. Based on the MSTAR dataset, the experimental results show that the method proposed achieves an average precision of 88.95% and the training convergence rate is significantly improved in small sample scenarios. [ABSTRACT FROM AUTHOR]

The recognition and monitoring of marine ranching help to determine the type, spatial distribution and dynamic change of marine life, as well as promote the rational utilization of marine resources and marine ecological environment protection, which has important research significance and application value. In the study of marine ranching recognition based on high-resolution remote sensing images, the convolutional neural network (CNN)-based deep learning method, which can adapt to different water environments, can be used to identify multi-scale targets and restore the boundaries of extraction results with high accuracy. However, research based on deep learning still has problems such as rarely considering the feature complexity of marine ranching and inadequate discussion of model generalization for multi-temporal images. In this paper, we construct a multi-temporal dataset that describes multiple features of marine ranching to explore the recognition and generalization ability of the semantic segmentation models DeepLab-v3+ and U-Net (used in large-area marine ranching) based on GF-1 remote sensing images with a 2 m spatial resolution. Through experiments, we find that the F-score of the U-Net extraction results from multi-temporal test images is basically stable at more than 90%, while the F-score of DeepLab-v3+ fluctuates around 80%. The results show that, compared with DeepLab-v3+, U-Net has a stronger recognition and generalization ability for marine ranching. U-Net can identify floating raft aquaculture areas at different growing stages and distinguish between cage aquaculture areas with different colors but similar morphological characteristics. [ABSTRACT FROM AUTHOR]

Ship detection in synthetic aperture radar (SAR) images has witnessed rapid development in recent years, especially after the adoption of convolutional neural network (CNN)-based methods. Recently, a transformer using self-attention and a feed forward neural network with a encoder-decoder structure has received much attention from researchers, due to its intrinsic characteristics of global-relation modeling between pixels and an enlarged global receptive field. However, when adapting transformers to SAR ship detection, one challenging issue cannot be ignored. Background clutter, such as a coast, an island, or a sea wave, made previous object detectors easily miss ships with a blurred contour. Therefore, in this paper, we propose a local-sparse-information-aggregation transformer with explicit contour guidance for ship detection in SAR images. Based on the Swin Transformer architecture, in order to effectively aggregate sparse meaningful cues of small-scale ships, a deformable attention mechanism is incorporated to change the original self-attention mechanism. Moreover, a novel contour-guided shape-enhancement module is proposed to explicitly enforce the contour constraints on the one-dimensional transformer architecture. Experimental results show that our proposed method achieves superior performance on the challenging HRSID and SSDD datasets. [ABSTRACT FROM AUTHOR]

Conventional practices of bridge visual inspection present several limitations, including a tedious process of analyzing images manually to identify potential damages. Vision-based techniques, particularly Deep Convolutional Neural Networks, have been widely investigated to automatically identify, localize, and quantify defects in bridge images. However, massive datasets with different annotation levels are required to train these deep models. This paper presents a dataset of more than 6900 images featuring three common defects of concrete bridges (i.e., cracks, efflorescence, and spalling). To overcome the challenge of limited training samples, three Transfer Learning approaches in fine-tuning the state-of-the-art Visual Geometry Group network were studied and compared to classify the three defects. The best-proposed approach achieved a high testing accuracy (97.13%), combined with high F1-scores of 97.38%, 95.01%, and 97.35% for cracks, efflorescence, and spalling, respectively. Furthermore, the effectiveness of interpretable networks was explored in the context of weakly supervised semantic segmentation using image-level annotations. Two gradient-based backpropagation interpretation techniques were used to generate pixel-level heatmaps and localize defects in test images. Qualitative results showcase the potential use of interpretation maps to provide relevant information on defect localization in a weak supervision framework. [ABSTRACT FROM AUTHOR]

Change detection (CD) in hyperspectral images has become a research hotspot in the field of remote sensing due to the extremely wide spectral range of hyperspectral images compared to traditional remote sensing images. It is challenging to effectively extract features from redundant high-dimensional data for hyperspectral change detection tasks due to the fact that hyperspectral data contain abundant spectral information. In this paper, a novel feature extraction network is proposed, which uses a Recurrent Neural Network (RNN) to mine the spectral information of the input image and combines this with a Convolutional Neural Network (CNN) to fuse the spatial information of hyperspectral data. Finally, the feature extraction structure of hybrid RNN and CNN is used as a building block to complete the change detection task. In addition, we use an unsupervised sample generation strategy to produce high-quality samples for network training. The experimental results demonstrate that the proposed method yields reliable detection results. Moreover, the proposed method has fewer noise regions than the pixel-based method. [ABSTRACT FROM AUTHOR]

Artificial reef detection in multibeam sonar images is an important measure for the monitoring and assessment of biological resources in marine ranching. With respect to how to accurately detect artificial reefs in multibeam sonar images, this paper proposes an artificial reef detection framework for multibeam sonar images based on convolutional neural networks (CNN). First, a large-scale multibeam sonar image artificial reef detection dataset, FIO-AR, was established and made public to promote the development of artificial multibeam sonar image artificial reef detection. Then, an artificial reef detection framework based on CNN was designed to detect the various artificial reefs in multibeam sonar images. Using the FIO-AR dataset, the proposed method is compared with some state-of-the-art artificial reef detection methods. The experimental results show that the proposed method can achieve an 86.86% F1-score and a 76.74% intersection-over-union (IOU) and outperform some state-of-the-art artificial reef detection methods. [ABSTRACT FROM AUTHOR]

The development of convolution neural networks (CNNs) has become a significant means to solve the problem of remote sensing scene image classification. However, well-performing CNNs generally have high complexity and are prone to overfitting. To handle the above problem, we present a new classification approach using an mmsCNN–HMM combined model with stacking ensemble mechanism in this paper. First of all, a modified multi-scale convolution neural network (mmsCNN) is proposed to extract multi-scale structural features, which has a lightweight structure and can avoid high computational complexity. Then, we utilize a hidden Markov model (HMM) to mine the context information of the extracted features of the whole sample image. For different categories of scene images, the corresponding HMM is trained and all the trained HMMs form an HMM group. In addition, our approach is based on a stacking ensemble learning scheme, in which the preliminary predicted values generated by the HMM group are used in an extreme gradient boosting (XGBoost) model to generate the final prediction. This stacking ensemble learning mechanism integrates multiple models to make decisions together, which can effectively prevent overfitting while ensuring accuracy. Finally, the trained XGBoost model conducts the scene category prediction. In this paper, the six most widely used remote sensing scene datasets, UCM, RSSCN, SIRI-WHU, WHU-RS, AID, and NWPU, are selected to carry out all kinds of experiments. The numerical experiments verify that the proposed approach shows more important advantages than the advanced approaches. [ABSTRACT FROM AUTHOR]

High-frequency surface wave radar (HFSWR) is an important marine monitoring technology, and this new regime of radar plays an important role in large-scale, continuous early-warning monitoring at sea. In particular, shipborne HFSWR has wider applications in detecting interesting sea areas, with the advantages of flexible deployment and extended detection capability. Due to the large amount of sea clutter accompanying the echo signals of shipborne HFSWR and the spread of sea clutter due to platform motion, the detection of targets in clutter regions is extremely difficult. In this paper, a multi-frame time-frequency (TF) analysis–based target-detection method is proposed. First, the sea clutter spreading area in the HFSWR echo signal is modeled, and the effects of platform motion and currents on the sea clutter spread are analyzed to determine the sea clutter coverage area; this paper focuses on frequency modeling. Then the TF image (TFI) of each range cell is obtained by TF analysis of the cells within a certain range of the echo signal, and the range cells of possible target points are determined by binary classification of the TFI through a convolutional neural network. Finally, the location of the final target point is obtained by correlation of multi-frame TFIs. Shipborne HFSWR field experiments show that the proposed detection method performs well in detecting targets concealed by sea clutter. [ABSTRACT FROM AUTHOR]

Superpixel segmentation is widely used in polarimetric synthetic aperture radar (PolSAR) image classification. However, the classification method using simple majority voting cannot easily handle evidence conflicts in a single superpixel. At present, there is no method to evaluate the quality of superpixel classification. To solve the above problems, this paper proposes a hybrid classification model based on superpixel entropy discrimination (SED), and constructs a two-level cascade classifier. Firstly, a light gradient boosting machine (LGBM) was used to process large-dimensional input features, and simple linear iterative clustering (SLIC) was integrated to obtain the primary classification results based on superpixels. Secondly, information entropy was introduced to evaluate the quality of superpixel classification, and a complex-valued convolutional neural network (CV-CNN) was used to reclassify the high-entropy superpixels to obtain the secondary classification results. Experiments with two measured PolSAR datasets show that the overall accuracy of both classification methods exceeded 97%. This method suppressed the evidence conflict in a single superpixel and the inaccuracy of superpixel segmentation. The test time of our proposed method was shorter than that of CV-CNN, and using only 55% of CV-CNN test data could achieve the same accuracy as using CV-CNN for the whole image. [ABSTRACT FROM AUTHOR]

Cloud segmentation is a fundamental step in accurately acquiring cloud cover. However, due to the nonrigid structures of clouds, traditional cloud segmentation methods perform worse than expected. In this paper, a novel deep convolutional neural network (CNN) named MA-SegCloud is proposed for segmenting cloud images based on a multibranch asymmetric convolution module (MACM) and an attention mechanism. The MACM is composed of asymmetric convolution, depth-separable convolution, and a squeeze-and-excitation module (SEM). The MACM not only enables the network to capture more contextual information in a larger area but can also adaptively adjust the feature channel weights. The attention mechanisms SEM and convolutional block attention module (CBAM) in the network can strengthen useful features for cloud image segmentation. As a result, MA-SegCloud achieves a 96.9% accuracy, 97.0% precision, 97.0% recall, 97.0% F-score, 3.1% error rate, and 94.0% mean intersection-over-union (MIoU) on the Singapore Whole-sky Nychthemeron Image Segmentation (SWINySEG) dataset. Extensive evaluations demonstrate that MA-SegCloud performs favorably against state-of-the-art cloud image segmentation methods. [ABSTRACT FROM AUTHOR]

Anomaly perception of infrared point targets has high application value in many fields, such as maritime surveillance, airspace surveillance, and early warning systems. This kind of abnormality includes the explosion of the target, the separation between stages, the disintegration caused by the abnormal strike, etc. By extracting the radiation characteristics of continuous frame targets, it is possible to analyze and warn the target state in time. Most anomaly detection methods adopt traditional outlier detection, which has the problems of poor accuracy and a high false alarm rate. Driven by data, this paper proposes a new network structure, called AC-LSTM, which combines Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM), and embeds the Periodic Time Series Data Attention module (PTSA). The network can better extract the spatial and temporal characteristics of one-dimensional time series data, and the PTSA module can consider the periodic characteristics of the target in the process of continuous movement, and focus on abnormal data. In addition, this paper also proposes a new time series data enhancement method, which slices and re-amplifies the long time series data. This method significantly improves the accuracy of anomaly detection. Through a large number of experiments, AC-LSTM has achieved higher scores on our collected datasets than other methods. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have achieved milestones in object detection of synthetic aperture radar (SAR) images. Recently, vision transformers and their variants have shown great promise in detection tasks. However, ship detection in SAR images remains a substantial challenge because of the characteristics of strong scattering, multi-scale, and complex backgrounds of ship objects in SAR images. This paper proposes an enhancement Swin transformer detection network, named ESTDNet, to complete the ship detection in SAR images to solve the above problems. We adopt the Swin transformer of Cascade-R-CNN (Cascade R-CNN Swin) as a benchmark model in ESTDNet. Based on this, we built two modules in ESTDNet: the feature enhancement Swin transformer (FESwin) module for improving feature extraction capability and the adjacent feature fusion (AFF) module for optimizing feature pyramids. Firstly, the FESwin module is employed as the backbone network, aggregating contextual information about perceptions before and after the Swin transformer model using CNN. It uses single-point channel information interaction as the primary and local spatial information interaction as the secondary for scale fusion based on capturing visual dependence through self-attention, which improves spatial-to-channel feature expression and increases the utilization of ship information from SAR images. Secondly, the AFF module is a weighted selection fusion of each high-level feature in the feature pyramid with its adjacent shallow-level features using learnable adaptive weights, allowing the ship information of SAR images to be focused on the feature maps at more scales and improving the recognition and localization capability for ships in SAR images. Finally, the ablation study conducted on the SSDD dataset validates the effectiveness of the two components proposed in the ESTDNet detector. Moreover, the experiments executed on two public datasets consisting of SSDD and SARShip demonstrate that the ESTDNet detector outperforms the state-of-the-art methods, which provides a new idea for ship detection in SAR images. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural network (CNN)-based methods have been extensively explored for synthetic aperture radar (SAR) target detection. Nevertheless, the convolutional sampling locations of CNNs cannot accurately fit vehicle targets due to the fixed sampling mechanism in the convolutional kernel. In this paper, we focus on the vehicle target detection task in SAR images and propose a novel rectangle-invariant rotatable convolution (RIRConv) to determine more accurately the convolutional sampling locations for vehicle targets. Specifically, this paper considers the shape characteristic of vehicle targets in SAR images, which always retain a rectangular shape despite having varying sizes, aspect ratios, and rotation angles. The proposed RIRConv equips three additional learnable attribute parameters, namely, width, height, and angle attributes, to adaptively adjust the sampling locations in the convolutional kernel according to the targets. In addition, the RIRConv applies a modulation mechanism to focus on the sampling locations that significantly affect the output. Finally, the RIRConv is introduced into the single-shot multibox detector (SSD) to realize SAR vehicle target detection. In this way, the feature representation capability of SSD for vehicle targets can be enhanced, thus leading to higher detection performance. Notably, the proposed RIRConv is "plug-and-play" and can also be used with other existing advanced technologies to achieve higher detection performance. The experiments based on the measured miniSAR data validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Small object detection in remote sensing enables the identification and analysis of unapparent but important information, playing a crucial role in various ground monitoring tasks. Due to the small size, the available feature information contained in small objects is very limited, making them more easily buried by the complex background. As one of the research hotspots in remote sensing, although many breakthroughs have been made, there still exist two significant shortcomings for the existing approaches: first, the down-sampling operation commonly used for feature extraction can barely preserve weak features of objects in a tiny size; second, the convolutional neural network methods have limitations in modeling global context to address cluttered backgrounds. To tackle these issues, a high-resolution network with transformer embedding parallel detection (HRTP-Net) is proposed in this paper. A high-resolution feature fusion network (HR-FFN) is designed to solve the first problem by maintaining high spatial resolution features with enhanced semantic information. Furthermore, a Swin-transformer-based mixed attention module (STMA) is proposed to augment the object information in the transformer block by establishing a pixel-level correlation, thereby enabling global background–object modeling, which can address the second shortcoming. Finally, a parallel detection structure for remote sensing is constructed by integrating the attentional outputs of STMA with standard convolutional features. The proposed method effectively mitigates the impact of the intricate background on small objects. The comprehensive experiment results on three representative remote sensing datasets with small objects (MASATI, VEDAI and DOTA datasets) demonstrate that the proposed HRTP-Net achieves a promising and competitive performance. [ABSTRACT FROM AUTHOR]

Semantic change detection (SCD) holds a critical place in remote sensing image interpretation, as it aims to locate changing regions and identify their associated land cover classes. Presently, post-classification techniques stand as the predominant strategy for SCD due to their simplicity and efficacy. However, these methods often overlook the intricate relationships between alterations in land cover. In this paper, we argue that comprehending the interplay of changes within land cover maps holds the key to enhancing SCD's performance. With this insight, a Temporal-Transform Module (TTM) is designed to capture change relationships across temporal dimensions. TTM selectively aggregates features across all temporal images, enhancing the unique features of each temporal image at distinct pixels. Moreover, we build a Temporal-Transform Network (TTNet) for SCD, comprising two semantic segmentation branches and a binary change detection branch. TTM is embedded into the decoder of each semantic segmentation branch, thus enabling TTNet to obtain better land cover classification results. Experimental results on the SECOND dataset show that TTNet achieves enhanced performance when compared to other benchmark methods in the SCD task. In particular, TTNet elevates mIoU accuracy by a minimum of 1.5% in the SCD task and 3.1% in the semantic segmentation task. [ABSTRACT FROM AUTHOR]

Hyperspectral sensors acquire spectral responses from objects with a large number of narrow spectral bands. The large volume of data may be costly in terms of storage and computational requirements. In addition, hyperspectral data are often information-wise redundant. Band selection intends to overcome these limitations by selecting a small subset of spectral bands that provide more information or better performance for particular tasks. However, existing band selection techniques do not directly maximize the task-specific performance, but rather utilize hand-crafted metrics as a proxy to the final goal of performance improvement. In this paper, we propose a deep learning (DL) architecture composed of a constrained measurement learning network for band selection, followed by a classification network. The proposed joint DL architecture is trained in a data-driven manner to optimize the classification loss along band selection. In this way, the proposed network directly learns to select bands that enhance the classification performance. Our evaluation results with Indian Pines (IP) and the University of Pavia (UP) datasets show that the proposed constrained measurement learning-based band selection approach provides higher classification accuracy compared to the state-of-the-art supervised band selection methods for the same number of bands selected. The proposed method shows 89.08 % and 97.78 % overall accuracy scores for IP and UP respectively, being 1.34 % and 2.19 % higher than the second-best method. [ABSTRACT FROM AUTHOR]