Search

The integration of multi-source remote sensing data, bolstered by advancements in deep learning, has emerged as a pivotal strategy for enhancing land use and land cover (LULC) classification accuracy. However, current methods often fail to consider the numerous prior knowledge of remote sensing images and the characteristics of heterogeneous remote sensing data, resulting in data loss between different modalities and the loss of a significant amount of useful information, thus affecting classification accuracy. To tackle these challenges, this paper proposes a LULC classification method based on remote sensing data that combines a Transformer and cross-pseudo-siamese learning deep neural network (TCPSNet). It first conducts shallow feature extraction in a dynamic multi-scale manner, fully leveraging the prior information of remote sensing data. Then, it further models deep features through the multimodal cross-attention module (MCAM) and cross-pseudo-siamese learning module (CPSLM). Finally, it achieves comprehensive fusion of local and global features through feature-level fusion and decision-level fusion combinations. Extensive experiments on datasets such as Trento, Houston 2013, Augsburg, MUUFL and Berlin demonstrate the superior performance of the proposed TCPSNet. The overall accuracy (OA) of the network on the Trento, Houston 2013 and Augsburg datasets is of 99.76%, 99.92%, 97.41%, 87.97% and 97.96%, respectively. [ABSTRACT FROM AUTHOR]

In recent years, the use of deep neural network in effective network feature extraction and the design of efficient and high-precision hyperspectral image classification algorithms has gradually become a research hotspot for scholars. However, due to the difficulty of obtaining hyperspectral images and the high cost of annotation, the training samples are very limited. In order to cope with the small sample problem, researchers often deepen the network model and use the attention mechanism to extract features; however, as the network model continues to deepen, the gradient disappears, the feature extraction ability is insufficient, and the computational cost is high. Therefore, how to make full use of the spectral and spatial information in limited samples has gradually become a difficult problem. In order to cope with such problems, this paper proposes two-branch multiscale spatial–spectral feature aggregation with a self-attention mechanism for a hyperspectral image classification model (FHDANet); the model constructs a dense two-branch pyramid structure, which can achieve the high efficiency extraction of joint spatial–spectral feature information and spectral feature information, reduce feature loss to a large extent, and strengthen the model's ability to extract contextual information. A channel–space attention module, ECBAM, is proposed, which greatly improves the extraction ability of the model for salient features, and a spatial information extraction module based on the deep feature fusion strategy HLDFF is proposed, which fully strengthens feature reusability and mitigates the feature loss problem brought about by the deepening of the model. Compared with five hyperspectral image classification algorithms, SVM, SSRN, A2S2K-ResNet, HyBridSN, SSDGL, RSSGL and LANet, this method significantly improves the classification performance on four representative datasets. Experiments have demonstrated that FHDANet can better extract and utilise the spatial and spectral information in hyperspectral images with excellent classification performance under small sample conditions. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing images (HSIs) have both spectral and spatial characteristics. The adept exploitation of these attributes is central to enhancing the classification accuracy of HSIs. In order to effectively utilize spatial and spectral features to classify HSIs, this paper proposes a method for the spatial feature extraction of HSIs based on a mutually guided image filter (muGIF) and combined with the band-distance-grouped principal component. Firstly, aiming at the problem that previously guided image filtering cannot effectively deal with the inconsistent information structure between the guided and target information, a method for extracting spatial features using muGIF is proposed. Then, aiming at the problem of the information loss caused by a single principal component as a guided image in the traditional GIF-based spatial–spectral classification, a spatial feature-extraction framework based on the band-distance-grouped principal component is proposed. The method groups the bands according to the band distance and extracts the principal components of each set of band subsets as the guide map of the current band subset to filter the HSIs. A deep convolutional neural network model and a generative adversarial network model for the filtered HSIs are constructed and then trained using samples for HSIs' spatial–spectral classification. Experiments show that compared with the traditional methods and several popular spatial–spectral HSI classification methods based on a filter, the proposed methods based on muGIF can effectively extract the spatial–spectral features and improve the classification accuracy of HSIs. [ABSTRACT FROM AUTHOR]

The accurate semantic segmentation of point cloud data is the basis for their application in the inspection of extra high-voltage transmission lines (EHVTL). As deep learning evolves, point-wise-based deep neural networks have shown great potential for the semantic segmentation of EHVTL point clouds. However, EHVTL point cloud data are characterized by a large data volume and significant class imbalance. Therefore, the down-sampling method and point cloud feature extraction method used in current point-wise-based deep neural networks hardly meet the needs of computational accuracy and efficiency. In this paper, we proposed a two-step down-sampling method and a point cloud feature extraction method based on local feature aggregation of the point clouds after down-sampling in each layer of the model (LFAPAD). We then established a deep neural network named PowerLine-Net for the semantic segmentation of the EHVTL point clouds. Furthermore, in order to test and analyze the performance of PowerLine-Net, we constructed a point cloud dataset for the EHVTL scenes. Using this dataset and the Semantic3D dataset, we implemented network parameter testing, semantic segmentation, and an accuracy comparison of different networks based on PowerLine-Net. The results illustrate that the semantic segmentation model proposed in this paper has a high computational efficiency and accuracy in the semantic segmentation of EHVTL point clouds. Compared with conventional deep neural networks, including PointCNN, KPConv, SPG, PointNet++, and RandLA-Net, PowerLine-Net also achieves a higher accuracy in the semantic segmentation of EHVTL point clouds. Moreover, based on the results predicted by PowerLine-Net, the risk point detection for EHVTL point clouds has been achieved, which demonstrates the important value of this network in practical applications. In addition, as shown by the results of Semantic3D, PowerLine-Net also achieves a high segmentation accuracy, which proves its powerful capability and wide applicability in semantic segmentation for the point clouds of large-scale scenes. [ABSTRACT FROM AUTHOR]

Remote sensing change detection involves detecting pixels that have changed from a bi-temporal image of the same location. Current mainstream change detection models use encoder-decoder structures as well as Siamese networks. However, there are still some challenges with this: (1) Existing change feature fusion approaches do not take into account the symmetry of change features, which leads to information loss; (2) The encoder is independent of the change detection task, and feature extraction is performed separately for dual-time images, which leads to underutilization of the encoder parameters; (3) There are problems of unbalanced positive and negative samples and bad edge region detection. To solve the above problems, a mutual feature-aware network (MFNet) is proposed in this paper. Three modules are proposed for the purpose: (1) A symmetric change feature fusion module (SCFM), which uses double-branch feature selection without losing feature information and focuses explicitly on focal spatial regions based on cosine similarity to introduce strong a priori information; (2) A mutual feature-aware module (MFAM), which introduces change features in advance at the encoder stage and uses a cross-type attention mechanism for long-range dependence modeling; (3) A loss function for edge regions. After detailed experiments, the F1 scores of MFNet on SYSU-CD and LEVIR-CD were 83.11% and 91.52%, respectively, outperforming several advanced algorithms, demonstrating the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Marine oil spills can cause serious damage to marine ecosystems and biological species, and the pollution is difficult to repair in the short term. Accurate oil type identification and oil thickness quantification are of great significance for marine oil spill emergency response and damage assessment. In recent years, hyperspectral remote sensing technology has become an effective means to monitor marine oil spills. The spectral and spatial features of oil spill images at different levels are different. To accurately identify oil spill types and quantify oil film thickness, and perform better extraction of spectral and spatial features, a multilevel spatial and spectral feature extraction network is proposed in this study. First, the graph convolutional neural network and graph attentional neural network models were used to extract spectral and spatial features in non-Euclidean space, respectively, and then the designed modules based on 2D expansion convolution, depth convolution, and point convolution were applied to extract feature information in Euclidean space; after that, a multilevel feature fusion method was developed to fuse the obtained spatial and spectral features in Euclidean space in a complementary way to obtain multilevel features. Finally, the multilevel features were fused at the feature level to obtain the oil spill information. The experimental results show that compared with CGCNN, SSRN, and A2S2KResNet algorithms, the accuracy of oil type identification and oil film thickness classification of the proposed method in this paper is improved by 12.82%, 0.06%, and 0.08% and 2.23%, 0.69%, and 0.47%, respectively, which proves that the method in this paper can effectively extract oil spill information and identify different oil spill types and different oil film thicknesses. [ABSTRACT FROM AUTHOR]

In recent years, data-driven deep networks have demonstrated remarkable detection performance for infrared small targets. However, continuously increasing the depth of neural networks to enhance performance has proven impractical. Consequently, the integration of prior physical knowledge related to infrared small targets within deep neural networks has become crucial. It aims to improve the models' awareness of inherent physical characteristics. In this paper, we propose a novel dual-domain prior-driven deep network (DPDNet) for infrared small-target detection. Our method integrates the advantages of both data-driven and model-driven methods by leveraging the prior physical characteristics as the driving force. Initially, we utilize the sparse characteristics of infrared small targets to boost their saliency at the input level of the network. Subsequently, a high-frequency feature extraction module, seamlessly integrated into the network's backbone, is employed to excavate feature information. DPDNet simultaneously emphasizes the prior sparse characteristics of infrared small targets in the spatial domain and their prior high-frequency characteristics in the frequency domain. Compared with previous CNN-based methods, our method achieves superior performance while utilizing fewer convolutional layers. It has a performance of 78.64% IoU, 95.56 Pd, and 2.15 × 10−6 Fa on the SIRST dataset. [ABSTRACT FROM AUTHOR]

With increasing access to open spatial data, it is possible to improve the quality of analyses carried out in the preliminary stages of the investment process. The extraction of buildings from raster data is an important process, especially for urban, planning and environmental studies. It allows, after processing, to represent buildings registered on a given image, e.g., in a vector format. With an actual image it is possible to obtain current information on the location of buildings in a defined area. At the same time, in recent years, there has been huge progress in the use of machine learning algorithms for object identification purposes. In particular, the semantic segmentation algorithms of deep convolutional neural networks which are based on the extraction of features from an image by means of masking have proven themselves here. The main problem with the application of semantic segmentation is the limited availability of masks, i.e., labelled data for training the network. Creating datasets based on manual labelling of data is a tedious, time consuming and capital-intensive process. Furthermore, any errors may be reflected in later analysis results. Therefore, this paper aims to show how to automate the process of data labelling of cadastral data from open spatial databases using convolutional neural networks, and to identify and extract buildings from high resolution orthophotomaps based on this data. The conducted research has shown that automatic feature extraction using semantic ML segmentation on the basis of data from open spatial databases is possible and can provide adequate quality of results. [ABSTRACT FROM AUTHOR]

Hyperspectral anomaly detection is a popular research direction for hyperspectral images; however, it is problematic because it separates the background and anomaly without prior target information. Currently, deep neural networks are used as an extractor to mine intrinsic features in hyperspectral images, which can be fed into separate anomaly detection methods to improve their performances. However, this hybrid approach is suboptimal because the subsequent detector is unable to drive the data representation in hidden layers, which makes it a challenge to maximize the capabilities of deep neural networks when extracting the underlying features customized for anomaly detection. To address this issue, a novel unsupervised, self-attention-based, one-class neural network (SAOCNN) is proposed in this paper. SAOCNN consists of two components: a novel feature extraction network and a one-class SVM (OC-SVM) anomaly detection method, which are interconnected and jointly trained by the OC-SVM-like loss function. The adoption of co-training updates the feature extraction network together with the anomaly detector, thus improving the whole network's detection performance. Considering that the prominent feature of an anomaly lies in its difference from the background, we designed a deep neural extraction network to learn more comprehensive hyperspectral image features, including spectral, global correlation, and local spatial features. To accomplish this goal, we adopted an adversarial autoencoder to produce the residual image with highlighted anomaly targets and a suppressed background, which is input into an improved non-local module to adaptively select the useful global information in the whole deep feature space. In addition, we incorporated a two-layer convolutional network to obtain local features. SAOCNN maps the original hyperspectral data to a learned feature space with better anomaly separation from the background, making it possible for the hyperplane to separate them. Our experiments on six public hyperspectral datasets demonstrate the state-of-the-art performance and superiority of our proposed SAOCNN when extracting deep potential features, which are more conducive to anomaly detection. [ABSTRACT FROM AUTHOR]

Loop closure detection is an important component of Simultaneous Localization and Mapping (SLAM). In this paper, a novel two-branch loop closure detection algorithm unifying deep Convolutional Neural Network (ConvNet) features and semantic edge features is proposed. In detail, we use one feature extraction module to extract both ConvNet and semantic edge features simultaneously. The deep ConvNet features are subjected to a Context Feature Enhancement (CFE) module in the global feature ranking branch to generate a representative global feature descriptor. Concurrently, to reduce the interference of dynamic features, the extracted semantic edge information of landmarks is encoded through the Vector of Locally Aggregated Descriptors (VLAD) framework in the semantic edge feature ranking branch to form semantic edge descriptors. Finally, semantic, visual, and geometric information is integrated by the similarity score fusion calculation. Extensive experiments on six public datasets show that the proposed approach can achieve competitive recall rates at 100% precision compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Marine activities occupy an important position in human society. The accurate classification of ships is an effective monitoring method. However, traditional image classification has the problem of low classification accuracy, and the corresponding ship dataset also has the problem of long-tail distribution. Aimed at solving these problems, this paper proposes a fine-grained classification method of optical remote sensing ship images based on deep convolution neural network. We use three-level images to extract three-level features for classification. The first-level image is the original image as an auxiliary. The specific position of the ship in the original image is located by the gradient-weighted class activation mapping. The target-level image as the second-level image is obtained by threshold processing the class activation map. The third-level image is the midship position image extracted from the target image. Then we add self-calibrated convolutions to the feature extraction network to enrich the output features. Finally, the class imbalance is solved by reweighting the class-balanced loss function. Experimental results show that we can achieve accuracies of 92.81%, 93.54% and 93.97%, respectively, after applying the proposed method on different datasets. Compared with other classification methods, this method has a higher accuracy in optical aerospace remote sensing ship classification. [ABSTRACT FROM AUTHOR]

In the last decade, unmanned aerial vehicle (UAV) images have become one of the most critical remote sensing data sources because of the high flexibility of UAV platforms and the extensive usage of low-cost cameras. For robust and accurate image orientation, Huang et al. [[2]] proposed a camera self-calibration solution for long-corridor UAV images, such as transmission lines. 10.3390/rs13163247 2 Huang W., Jiang S., Jiang W. Camera Self-Calibration with GNSS Constrained Bundle Adjustment for Weakly Structured Long Corridor UAV Images. [Extracted from the article]

Three-dimensional (3D) point cloud classification methods based on deep learning have good classification performance; however, they adapt poorly to diverse datasets and their classification accuracy must be improved. Therefore, FSDCNet, a neural network model based on the fusion of static and dynamic convolution, is proposed and applied for multiview 3D point cloud classification in this paper. FSDCNet devises a view selection method with fixed and random viewpoints, which effectively avoids the overfitting caused by the traditional fixed viewpoint. A local feature extraction operator of dynamic and static convolution adaptive weight fusion was designed to improve the model's adaptability to different types of datasets. To address the problems of large parameters and high computational complexity associated with the current methods of dynamic convolution, a lightweight and adaptive dynamic convolution operator was developed. In addition, FSDCNet builds a global attention pooling, integrating the most crucial information on different view features to the greatest extent. Due to these characteristics, FSDCNet is more adaptable, can extract more fine-grained detailed information, and can improve the classification accuracy of point cloud data. The proposed method was applied to the ModelNet40 and Sydney Urban Objects datasets. In these experiments, FSDCNet outperformed its counterparts, achieving state-of-the-art point cloud classification accuracy. For the ModelNet40 dataset, the overall accuracy (OA) and average accuracy (AA) of FSDCNet in a single view reached 93.8% and 91.2%, respectively, which were superior to those values for many other methods using 6 and 12 views. FSDCNet obtained the best results for 6 and 12 views, achieving 94.6%, 93.3%, 95.3%, and 93.6% in OA and AA metrics, respectively. For the Sydney Urban Objects dataset, FSDCNet achieved an OA and F1 score of 81.2% and 80.1% in a single view, respectively, which were higher than most of the compared methods. In 6 and 12 views, FSDCNet reached an OA of 85.3% and 83.6% and an F1 score of 85.5% and 83.7%, respectively. [ABSTRACT FROM AUTHOR]

Visual object tracking for unmanned aerial vehicles (UAV) is widely used in many fields such as military reconnaissance, search and rescue work, film shooting, and so on. However, the performance of existing methods is still not very satisfactory due to some complex factors including viewpoint changing, background clutters and occlusion. The Siamese trackers, which offer a convenient way of formulating the visual tracking problem as a template matching process, have achieved success in recent visual tracking datasets. Unfortunately, these template match-based trackers cannot adapt well to frequent appearance change in UAV video datasets. To deal with this problem, this paper proposes a template-driven Siamese network (TDSiam), which consists of feature extraction subnetwork, feature fusion subnetwork and bounding box estimation subnetwork. Especially, a template library branch is proposed for the feature extraction subnetwork to adapt to the changeable appearance of the target. In addition, a feature aligned (FA) module is proposed as the core of feature fusion subnetwork, which can fuse information in the form of center alignment. More importantly, a method for occlusion detection is proposed to reduce the noise caused by occlusion. Experiments were conducted on two challenging benchmarks UAV123 and UAV20L, the results verified the more competitive performance of our proposed method compared to the existing algorithms. [ABSTRACT FROM AUTHOR]

Point cloud registration is a critical problem because it is the basis of many 3D vision tasks. With the popularity of deep learning, many scholars have focused on leveraging deep neural networks to address the point cloud registration problem. However, many of these methods are still sensitive to partial overlap and differences in density distribution. For this reason, herein, we propose a method based on rotation-invariant features and using a sparse-to-dense matching strategy for robust point cloud registration. Firstly, we encode raw points as superpoints with a network combining KPConv and FPN, and their associated features are extracted. Then point pair features of these superpoints are computed and embedded into the transformer to learn the hybrid features, which makes the approach invariant to rigid transformation. Subsequently, a sparse-to-dense matching strategy is designed to address the registration problem. The correspondences of superpoints are obtained via sparse matching and then propagated to local dense points and, further, to global dense points, the byproduct of which is a series of transformation parameters. Finally, the enhanced features based on spatial consistency are repeatedly fed into the sparse-to-dense matching module to rebuild reliable correspondence, and the optimal transformation parameter is re-estimated for final alignment. Our experiments show that, with the proposed method, the inlier ratio and registration recall are effectively improved, and the performance is better than that of other point cloud registration methods on 3DMatch and ModelNet40. [ABSTRACT FROM AUTHOR]

Target detection in hyperspectral images (HSIs) aims to distinguish target pixels from the background using knowledge gleaned from prior spectra. Most traditional methods are based on certain assumptions and utilize handcrafted classifiers. These simple models and assumptions' failure restrict the detection performance under complicated background interference. Recently, based on the convolutional networks, many supervised deep learning detectors have outperformed the traditional methods. However, these methods suffer from unstable detection, heavy computation burden, and optimization difficulty. This paper proposes a Siamese fully connected based target detector (SFCTD) that comprises nonlinear feature extraction modules (NFEMs) and cosine distance classifiers. Two NFEMs, which extract discriminative spectral features of input spectra-pairs, are based on fully connected layers for efficient computing and share the parameters to ease the optimization. To solve the few samples problem, we propose a pseudo data generation method based on the linear mixed model and the assumption that background pixels are dominant in HSIs. For mitigating the impact of stochastic suboptimal initialization, we parallelly optimize several Siamese detectors with small computation burdens and aggregate them as ensembles in the inference time. The network ensembles outperform every detector in terms of stability and achieve an outstanding balance between background suppression and detection rate. Experiments on multiple data sets demonstrate that the proposed detector is superior to the state-of-the-art detectors. [ABSTRACT FROM AUTHOR]

Change detection for remote sensing images is an indispensable procedure for many remote sensing applications, such as geological disaster assessment, environmental monitoring, and urban development monitoring. Through this technique, the difference in certain areas after some emergencies can be determined to estimate their influence. Additionally, by analyzing the sequential difference maps, the change tendency can be found to help to predict future changes, such as urban development and environmental pollution. The complex variety of changes and interferential changes caused by imaging processing, such as season, weather and sensors, are critical factors that affect the effectiveness of change detection methods. Recently, there have been many research achievements surrounding this topic, but a perfect solution to all the problems in change detection has not yet been achieved. In this paper, we mainly focus on reducing the influence of imaging processing through the deep neural network technique with limited labeled samples. The attention-guided Siamese fusion network is constructed based on one basic Siamese network for change detection. In contrast to common processing, besides high-level feature fusion, feature fusion is operated during the whole feature extraction process by using an attention information fusion module. This module can not only realize the information fusion of two feature extraction network branches, but also guide the feature learning network to focus on feature channels with high importance. Finally, extensive experiments were performed on three public datasets, which could verify the significance of information fusion and the guidance of the attention mechanism during feature learning in comparison with related methods. [ABSTRACT FROM AUTHOR]

Predicting sea wave parameters such as significant wave height (SWH) has recently been identified as a critical requirement for maritime security and economy. Earth observation satellite missions have resulted in a massive rise in marine data volume and dimensionality. Deep learning technologies have proven their capabilities to process large amounts of data, draw useful insights, and assist in environmental decision making. In this study, a new deep-learning-based hybrid feature selection approach is proposed for SWH prediction using satellite Synthetic Aperture Radar (SAR) mode altimeter data. The introduced approach integrates the power of autoencoder deep neural networks in mapping input features into representative latent-space features with the feature selection power of the principal component analysis (PCA) algorithm to create significant features from altimeter observations. Several hybrid feature sets were generated using the proposed approach and utilized for modeling SWH using Gaussian Process Regression (GPR) and Neural Network Regression (NNR). SAR mode altimeter data from the Sentinel-3A mission calibrated by in situ buoy data was used for training and evaluating the SWH models. The significance of the autoencoder-based feature sets in improving the prediction performance of SWH models is investigated against original, traditionally selected, and hybrid features. The autoencoder–PCA hybrid feature set generated by the proposed approach recorded the lowest average RMSE values of 0.11069 for GPR models, which outperforms the state-of-the-art results. The findings of this study reveal the superiority of the autoencoder deep learning network in generating latent features that aid in improving the prediction performance of SWH models over traditional feature extraction methods. [ABSTRACT FROM AUTHOR]

Reliable quantitative precipitation forecasting is essential to society. At present, quantitative precipitation forecasting based on weather radar represents an urgently needed, yet rather challenging. However, because the Z-R relation between radar and rainfall has several parameters in different areas, and because rainfall varies with seasons, traditional methods cannot capture high-resolution spatiotemporal features. Therefore, we propose an attention fusion spatiotemporal residual network (AF-SRNet) to forecast rainfall precisely for the weak continuity of convective precipitation. Specifically, the spatiotemporal residual network is designed to extract the deep spatiotemporal features of radar echo and precipitation data. Then, we combine the radar echo feature and precipitation feature as the input of the decoder through the attention fusion block; after that, the decoder forecasts the rainfall for the next two hours. We train and evaluate our approaches on the historical data from the Jiangsu Meteorological Observatory. The experimental results show that AF-SRNet can effectively utilize multiple inputs and provides more precise nowcasting of convective precipitation. [ABSTRACT FROM AUTHOR]

Plants diseases constitute a substantial threat for farmers given the high economic and environmental impact of their treatment. Detecting possible pathogen threats in plants based on non-destructive remote sensing and computer vision methods offers an alternative to existing laboratory methods and leads to improved crop management. Vine is an important crop that is mainly affected by fungal diseases. In this study, photos from healthy leaves and leaves infected by a fungal disease of vine are used to create disease identification classifiers. The transfer learning technique was employed in this study and was used to train three different deep convolutional neural network (DCNN) approaches that were compared according to their classification accuracy, namely AlexNet, VGG-19, and Inception v3. The above-mentioned models were trained on the open-source PlantVillage dataset using two training approaches: feature extraction, where the weights of the base deep neural network model were frozen and only the ones on the newly added layers were updated, and fine tuning, where the weights of the base model were also updated during training. Then, the created models were validated on the PlantVillage dataset and retrained using a custom field-grown vine photo dataset. The results showed that the fine-tuning approach showed better validation and testing accuracy, for all DCNNs, compared to the feature extraction approach. As far as the results of DCNNs are concerned, the Inception v3 algorithm outperformed VGG-19 and AlexNet in almost all the cases, demonstrating a validation performance of 100% for the fine-tuned strategy on the PlantVillage dataset and an accuracy of 83.3% for the respective strategy on a custom vine disease use case dataset, while AlexNet achieved 87.5% validation and 66.7% accuracy for the respective scenarios. Regarding VGG-19, the validation performance reached 100%, with an accuracy of 76.7%. [ABSTRACT FROM AUTHOR]

In recent years, methods based on deep convolutional neural networks (CNNs) have dominated the classification task of hyperspectral images. Although CNN-based HSI classification methods have the advantages of spatial feature extraction, HSI images are characterized by approximately continuous spectral information, usually containing hundreds of spectral bands. CNN cannot mine and represent the sequence properties of spectral features well, and the transformer model of attention mechanism proves its advantages in processing sequence data. This study proposes a new spectral spatial kernel combined with the improved Vision Transformer (ViT) to jointly extract spatial spectral features to complete classification task. First, the hyperspectral data are dimensionally reduced by PCA; then, the shallow features are extracted with an spectral spatial kernel, and the extracted features are input into the improved ViT model. The improved ViT introduces a re-attention mechanism and a local mechanism based on the original ViT. The re-attention mechanism can increase the diversity of attention maps at different levels. The local mechanism is introduced into ViT to make full use of the local and global information of the data to improve the classification accuracy. Finally, a multi-layer perceptron is used to obtain the classification result. Among them, the Focal Loss function is used to increase the loss weight of small-class samples and difficult-to-classify samples in HSI data samples and reduce the loss weight of easy-to-classify samples, so that the network can learn more useful hyperspectral image information. In addition, using the Apollo optimizer to train the HSI classification model to better update and compute network parameters that affect model training and model output, thereby minimizing the loss function. We evaluated the classification performance of the proposed method on four different datasets, and achieved good classification results on urban land object classification, crop classification and mineral classification, respectively. Compared with the state-of-the-art backbone network, the method achieves a significant improvement and achieves very good classification accuracy. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing image (HSI) classification is very useful in different applications, and recently, deep learning has been applied for HSI classification successfully. However, the number of training samples is usually limited, causing difficulty in use of very deep learning models. We propose a wide and deep Fourier network to learn features efficiently by using pruned features extracted in the frequency domain. It is composed of multiple wide Fourier layers to extract hierarchical features layer-by-layer efficiently. Each wide Fourier layer includes a large number of Fourier transforms to extract features in the frequency domain from a local spatial area using sliding windows with given strides.These extracted features are pruned to retain important features and reduce computations. The weights in the final fully connected layers are computed using least squares. The transform amplitudes are used for nonlinear processing with pruned features. The proposed method was evaluated with HSI datasets including Pavia University, KSC, and Salinas datasets. The overall accuracies (OAs) of the proposed method can reach 99.77%, 99.97%, and 99.95%, respectively. The average accuracies (AAs) can achieve 99.55%, 99.95%, and 99.95%, respectively. The Kappa coefficients are as high as 99.69%, 99.96%, and 99.94%, respectively. The experimental results show that the proposed method achieved excellent performance among other compared methods. The proposed method can be used for applications including classification, and image segmentation tasks, and has the ability to be implemented with lightweight embedded computing platforms. The future work is to improve the method to make it available for use in applications including object detection, time serial data prediction, and fast implementation. [ABSTRACT FROM AUTHOR]

In this work, we construct a Kd–Octree hybrid index structure to organize the point cloud and generate patch-based feature descriptors at its leaf nodes. We propose a simple yet effective convolutional neural network, termed KdO-Net, with Kd–Octree based descriptors as input for 3D pairwise point cloud matching. The classic pipeline of 3D point cloud registration involves two steps, viz., the point feature matching and the globally consistent refinement. We focus on the first step that can be further divided into three parts, viz., the key point detection, feature descriptor extraction, and pairwise-point correspondence estimation. In practical applications, the point feature matching is ambiguous and challenging owing to the low overlap of multiple scans, inconsistency of point density, and unstructured properties. To solve these issues, we propose the KdO-Net for 3D pairwise point feature matching and present a novel nearest neighbor searching strategy to address the computation problem. Thereafter, our method is evaluated with respect to an indoor BundleFusion benchmark, and generalized to a challenging outdoor ETH dataset. Further, we have extended our method over our complicated and low-overlapped TUM-lab dataset. The empirical results graphically demonstrate that our method achieves a superior precision and a comparable feature matching recall to the prior state-of-the-art deep learning-based methods, despite the overlap being less than 30 percent. Finally, we implement quantitative and qualitative ablated experiments and visualization interpretations for illustrating the insights and behavior of our network. [ABSTRACT FROM AUTHOR]

Due to end-to-end optimization characteristics and fine generalization ability, convolutional neural networks have been widely applied to hyperspectral image (HSI) classification, playing an irreplaceable role. However, previous studies struggle with two major challenges: (1) HSI contains complex topographic features, the number of labeled samples in different categories is unbalanced, resulting in poor classification for categories with few labeled samples; (2) With the deepening of neural network models, it is difficult to extract more discriminative spectral-spatial features. To address the issues mentioned above, we propose a discriminative spectral-spatial-semantic feature network based on shuffle and frequency attention mechanisms for HSI classification. There are four main parts of our approach: spectral-spatial shuffle attention module (SSAM), context-aware high-level spectral-spatial feature extraction module (CHSFEM), spectral-spatial frequency attention module (SFAM), and cross-connected semantic feature extraction module (CSFEM). First, to fully excavate the category attribute information, SSAM based on a "Deconstruction-Reconstruction" structure is designed, solving the problem of poor classification performance caused by an unbalanced number of label samples. Considering that deep spectral-spatial features are difficult to extract, CHSFEM and SFAM are constructed. The former is based on the "Horizontal-Vertical" structure to capture context-aware high-level multiscale features. The latter introduces multiple frequency components to compress channels to obtain more multifarious features. Finally, towards suppressing noisy boundaries efficiently and capturing abundant semantic information, CSFEM is devised. Numerous experiments are implemented on four public datasets: the evaluation indexes of OA, AA and Kappa on four datasets all exceed 99%, demonstrating that our method can achieve satisfactory performance and is superior to other contrasting methods. [ABSTRACT FROM AUTHOR]

The scattering features of objects in synthetic aperture radar (SAR) imagery are highly sensitive to different azimuth angles, and detecting azimuth-sensitive objects in complex scenes becomes a challenging task. To address this issue, we propose a novel framework called the spatial orientation attention enhancement network (SOAEN) by using aircraft detection in complex scenes of SAR imagery as a case study. Taking YOLOX as the basic framework, this framework introduces the inverted pyramid ConvMixer network (IPCN), the spatial-orientation-enhanced path aggregation feature pyramid network (SOEPAFPN), and the anchor-free decoupled head (AFDH) to achieve performance improvement. A spatial orientation attention module is proposed and introduced into the path aggregation feature pyramid network to form a new structure, the SOEPAFPN, for capturing feature transformations in different directions, highlighting object features and suppressing background effects; the IPCN is adapted to replace the backbone network of YOLOX for enhancing the multiscale feature extraction capability and reducing the computational complexity, while the AFDH is used to decouple object localization and classification to improve the efficiency and accuracy of object localization and classification. The experimental results of the multiple real complex scenes on Gaofen-3 1 m images show that the proposed method achieves the highest detection accuracy, with an average detection rate of 91.22% compared with the YOLO series networks. [ABSTRACT FROM AUTHOR]