Search

In power grid surveying, it is often necessary to fuse panchromatic and multispectral imagery for the design of power lines. Despite the abundance of deep learning networks for fusing these images, the results often suffer from spectral information loss or structural blurring. This study introduces a fusion model specifically tailored for power grid surveying that significantly enhances the representation of spatial–spectral features in remote sensing images. The model comprises three main modules: a TransforRS-Mamba module that integrates the sequence processing capabilities of the Mamba model with the attention mechanism of the Transformer to effectively merge spatial and spectral features; an improved spatial proximity-aware attention mechanism (SPPAM) that utilizes a spatial constraint matrix to greatly enhance the recognition of complex object relationships; and an optimized spatial proximity-constrained gated fusion module (SPCGF) that integrates spatial proximity constraints with residual connections to boost the recognition accuracy of key object features. To validate the effectiveness of the proposed method, extensive comparative and ablation experiments were conducted on GF-2 satellite images and the QuickBird (QB) dataset. Both qualitative and quantitative analyses indicate that our method outperforms 11 existing methods in terms of fusion effectiveness, particularly in reducing spectral distortion and spatial detail loss. However, the model's generalization performance across different data sources and environmental conditions has yet to be evaluated. Future research will explore the integration of various satellite datasets and assess the model's performance in diverse environmental contexts. [ABSTRACT FROM AUTHOR]

Hyperspectral target detection has a wide range of applications in marine target monitoring. Traditional methods for target detection take less consideration of the inherent structural information of hyperspectral images and make insufficient use of spatial information. These algorithms may experience degradation in efficacy during complex scenarios. To address these issues, this study introduces a hyperspectral target detection approach based on tensor adaptive reconstruction cascade spatial-spectral fusion, named as TRSSF. First, the position of the pixel that best matches the prior spectrum is obtained. Second, tensor decomposition and reconstruction of the original hyperspectral data are performed. Linear total variation smoothing is used to acquire the principal components in the spatial dimensionality unfolding of data, and correlation regularization robust principal component analysis is employed to derive the spectral dimensionality unfolding's principal components of data. Finally, the spatial-spectral fusion method is proposed for detecting hyperspectral targets on the reconstructed data. The use of multi-morphological feature fusion can fully utilize the spatial features to complement the spectral detection results and improve the integrity of target detection. The experiments conducted on the publicly available dataset and collected datasets demonstrated the effective detection achieved by the proposed method. [ABSTRACT FROM AUTHOR]

With the development of hyperspectral sensing technology, hyperspectral target detection technology plays an important role in remote target detection. However, existing hyperspectral target detection models are poorly adapted to complex backgrounds and mainly focus on the spectral domain, making less use of spatial structure information leading to low target detection rates. Therefore, a new target detection algorithm based on the prior spectral perception and local graph fusion is proposed. First, the prior spectrum-guided target extraction method is established. This method can take full advantage of the background and target spectral information by local inner and outer window linkage, reduce the impact of spectral variability on target acquisition performance, and improve detection stability. Second, the target enhancement strategy based on the Gabor multifeature graph is proposed. This technique makes full use of multidirectional and multiscale spatial information, which can reduce the influence of brightness, contrast and amplitude variation on detection performance due to light and angle. Finally, spatial–spectral fusion is executed to achieve target detection. It can make full use of spectral and spatial structure information to improve the target detection effect. Publicly available datasets and real collected datasets are adopted to check the validity of the proposed method. After comparison, it is found that the proposed algorithm has better detection effect than existing baseline methods. The maximum improvement in AUC values are 16.56%–88.16% across the eight datasets.

In power grid surveying, it is often necessary to fuse panchromatic and multispectral imagery for the design of power lines. Despite the abundance of deep learning networks for fusing these images, the results often suffer from spectral information loss or structural blurring. This study introduces a fusion model specifically tailored for power grid surveying that significantly enhances the representation of spatial–spectral features in remote sensing images. The model comprises three main modules: a TransforRS-Mamba module that integrates the sequence processing capabilities of the Mamba model with the attention mechanism of the Transformer to effectively merge spatial and spectral features; an improved spatial proximity-aware attention mechanism (SPPAM) that utilizes a spatial constraint matrix to greatly enhance the recognition of complex object relationships; and an optimized spatial proximity-constrained gated fusion module (SPCGF) that integrates spatial proximity constraints with residual connections to boost the recognition accuracy of key object features. To validate the effectiveness of the proposed method, extensive comparative and ablation experiments were conducted on GF-2 satellite images and the QuickBird (QB) dataset. Both qualitative and quantitative analyses indicate that our method outperforms 11 existing methods in terms of fusion effectiveness, particularly in reducing spectral distortion and spatial detail loss. However, the model’s generalization performance across different data sources and environmental conditions has yet to be evaluated. Future research will explore the integration of various satellite datasets and assess the model’s performance in diverse environmental contexts.

Marine oil spills pose a crucial concern in the monitoring of marine environments, and optical remote sensing serves as a vital means for marine oil spill detection. However, optical remote sensing imagery is susceptible to interference from sunglints and shadows, leading to diminished spectral differences between oil films and seawater. This makes it challenging to accurately extract the boundaries of oil–water interfaces. To address these aforementioned issues, this paper proposes a model based on the graph convolutional architecture and spatial–spectral information fusion for the oil spill detection of real oil spill incidents. The model is experimentally evaluated using both spaceborne and airborne hyperspectral oil spill images. Research findings demonstrate the superior oil spill detection accuracy of the developed model when compared to Graph Convolutional Network (GCN) and CNN-Enhanced Graph Convolutional Network (CEGCN), across two hyperspectral datasets collected from the Bohai Sea. Moreover, the performance of the developed model in oil spill detection remains optimal, even with only 1% of the training samples. Similar conclusions are drawn from the oil spill hyperspectral data collected from the Yellow Sea. These results validate the efficacy and robustness of the proposed model for marine oil spill detection. [ABSTRACT FROM AUTHOR]

Quantitative studies on surface elements require satellite hyperspectral images with high spatial resolution. The identification of different surface elements requires different characteristic bands and their corresponding optimal spatial–spectral fusion methods. To address these problems, the harmonic analysis (HA), guided filtering, and Gram–Schmidt (GS) algorithms were integrated to propose a spatial–spectral fusion method called HGF. The fusion experiment and validation of the hyperspectral images of GaoFen-5 (GF-5) and ZY1-02D were conducted separately using the HGF method, and the fusion effect was evaluated in three band intervals according to the spectral response of the ground class. First, HGF was used to fuse the GF-5 and GaoFen-1 (GF-1) images, and the fusion effect was evaluated both qualitatively and quantitatively. Second, the optimal fusion method was selected for the corresponding characteristic bands of the different surface elements. Finally, the hyperspectral image obtained by ZY1-02D and multispectral image of Sentinel-2B were used for validation to improve the accuracy and efficiency of satellite hyperspectral images in quantitative studies. The results show that for further studies on soil, vegetation, and water bodies, the best fusion methods in the 390–730, 730–1400, and 1400–2260 nm intervals are the GS, HGF, and HGF algorithms, respectively. Further analysis showed that the HGF or GS methods can be selected for quantitative studies on vegetation and water bodies and that the HGF method exhibits outstanding advantages for quantitative analysis of each soil element. [ABSTRACT FROM AUTHOR]

Marine oil spills pose a crucial concern in the monitoring of marine environments, and optical remote sensing serves as a vital means for marine oil spill detection. However, optical remote sensing imagery is susceptible to interference from sunglints and shadows, leading to diminished spectral differences between oil films and seawater. This makes it challenging to accurately extract the boundaries of oil–water interfaces. To address these aforementioned issues, this paper proposes a model based on the graph convolutional architecture and spatial–spectral information fusion for the oil spill detection of real oil spill incidents. The model is experimentally evaluated using both spaceborne and airborne hyperspectral oil spill images. Research findings demonstrate the superior oil spill detection accuracy of the developed model when compared to Graph Convolutional Network (GCN) and CNN-Enhanced Graph Convolutional Network (CEGCN), across two hyperspectral datasets collected from the Bohai Sea. Moreover, the performance of the developed model in oil spill detection remains optimal, even with only 1% of the training samples. Similar conclusions are drawn from the oil spill hyperspectral data collected from the Yellow Sea. These results validate the efficacy and robustness of the proposed model for marine oil spill detection.

Most of existing fusion methods generally directly apply both the LR-HSI and the HR-MSI as the inputs of the model for hyperspectral image fusion. However, this strategy will be difficult to fuse the two types of images well since the two type of inputs contain different features with significant differences in the scale. In this paper, we propose an efficient adaptive multi-scale input network (AMSIN) for HSI fusion. The proposed AMSIN contains adaptive multi-scale cross-mode input module, feature extraction&fusion module and image reconstruction module. In the adaptive multi-scale cross-mode input module, we design an adaptive multi-scale model to generate multiple input features with different scales to fully capture the multi-scale characteristics. Moreover, we introduce the cross-mode message insertion strategy to assign the two types of input features in suitable locations. In the feature extraction&fusion module, we fuse the multi-scale feature obtained from the adaptive multi-scale cross-mode input module by using the multi-scale spatial–spectral fusion block (MSSFB) to enhance the fusion performance. Then the HSI with high spatial resolution and spectral resolution is generated in the image reconstruction module. Extensive experiments on three datasets of Pavia University, Pavia Centre and Botswana demonstrate that the proposed AMSIN surpasses other nine state-of-the-art methods and performs best both in objective and subjective evaluations. [Display omitted] • We design an adaptive multi-scale cross-mode input module to better fuse HSI and MSI. • We design a multi-scale spatial-spectral fusion block to promote information fusion. • The visual and quantitative results show that AMSIN achieves the best fusion effect. [ABSTRACT FROM AUTHOR]

Joint sparse representation classification (JSRC) is a representative spectral–spatial classifier for hyperspectral images (HSIs). However, the JSRC is inappropriate for highly heterogeneous areas due to the spatial information being extracted from a fixed-sized neighborhood block, which is often unable to conform to the naturally irregular structure of land cover. To address this problem, a superpixel-based JSRC with nonlocal weighting, i.e., superpixel-based nonlocal weighted JSRC (SNLW-JSRC), is proposed in this paper. In SNLW-JSRC, the superpixel representation of an HSI is first constructed based on an entropy rate segmentation method. This strategy forms homogeneous neighborhoods with naturally irregular structures and alleviates the inclusion of pixels from different classes in the process of spatial information extraction. Afterwards, the superpixel-based nonlocal weighting (SNLW) scheme is built to weigh the superpixel based on its structural and spectral information. In this way, the weight of one specific neighboring pixel is determined by the local structural similarity between the neighboring pixel and the central test pixel. Then, the obtained local weights are used to generate the weighted mean data for each superpixel. Finally, JSRC is used to produce the superpixel-level classification. This speeds up the sparse representation and makes the spatial content more centralized and compact. To verify the proposed SNLW-JSRC method, we conducted experiments on four benchmark hyperspectral datasets, namely Indian Pines, Pavia University, Salinas, and DFC2013. The experimental results suggest that the SNLW-JSRC can achieve better classification results than the other four SRC-based algorithms and the classical support vector machine algorithm. Moreover, the SNLW-JSRC can also outperform the other SRC-based algorithms, even with a small number of training samples. [ABSTRACT FROM AUTHOR]

Deep learning methods have gained rapid development in hyperspectral pansharpening (HP) due to powerful spatial–spectral feature extraction ability. However, most of these methods are optimized using a single reconstruction objective. It is difficult for these methods to find a balance between spectral preservation and spatial preservation. Furthermore, these methods adopt interpolation or convolution to upsample the hyperspectral images (HSIs), which tends to cause noticeable spectral distortion. To conquer these issues, a novel multiobjective guided divide-and-conquer network (MO-DCN) is proposed for HP. It consists of a deconvolution long short-term memories (LSTMs) network (DLSTM) and a divide-and-conquer network (DCN). DLSTM leverages bi-direction learning to upsample HSIs by considering 3-D spatiotemporal dependencies. Then, DCN designs a two-branch architecture to reconstruct spatial and spectral information from upsampled HSIs and panchromatic images (PANIs), respectively, where the spatial branch designs an attention-in-attention module (AIAM) to emphasize complementary attention in a coarse-to-fine way. Finally, co-improvement of spatial and spectral information is formulated as an Epsilon-constraint-based multiobjective optimization. The Epsilon constraint method transforms one objective into a constraint and regards it as a penalty bound to make an excellent tradeoff between different objectives. Experimental results demonstrated that the proposed method markedly improves pansharpening performance in both the spatial and spectral domains and has superior fusion performance than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

High spatial resolution images have been widely applied to the monitoring of land surfaces during the past decades. However, satellite and unmanned aerial vehicle (UAV) sensors (e.g., IKONOS, and QuickBird) usually only provide visible and near-infrared spectral bands with a high spatial resolution. Such limited spectral bands at a high spatial resolution are not adequate for crucial environmental and agricultural studies that require essential bands (e.g., the red-edge (RE) and the short-wave infrared (SWIR)), such as land cover classification and the estimation of plant physiological parameters. On the contrary, abundant spectral information is commonly available on sensors with a medium spatial resolution (e.g., Landsat, and Sentinel-2). Existing spatial-spectral fusion methods can leverage the unique advantages of these two types of sensors, and produce synthesized images with both spatial and spectral detail. However, most spatial-spectral methods ignore the inherent geometric registration errors when using multi-source data, inevitably introducing the errors into the fusion result. To address the limitation, this study presents a new spatial-spectral fusion method, the Misregistration-Resistant Data Fusion approach (MRDF), to add new spectral bands to the high spatial resolution images from medium resolution images. The proposed method is composed of a local-model-based similar pixel searching and a global partial least square (PLS) regression, and has the following advantages: (1) the local model utilizes neighborhood similar pixels to generate predictions robust against geometric misregistration; (2) the combined model integrates a global PLS regression to improve fusion accuracy for the SWIR bands; (3) the extended-box strategy and spatial filtering can relieve geometric errors when minimizing residuals for the SWIR bands. Satellite and airborne data from three sites were used to compare the performance of MRDF with seven other typical methods. Results showed that MRDF not only produced accurate fusion results with above-average efficiency but also improved the classification accuracy, which substantiated its great potential for various applications.

Quantitative studies on surface elements require satellite hyperspectral images with high spatial resolution. The identification of different surface elements requires different characteristic bands and their corresponding optimal spatial–spectral fusion methods. To address these problems, the harmonic analysis (HA), guided filtering, and Gram–Schmidt (GS) algorithms were integrated to propose a spatial–spectral fusion method called HGF. The fusion experiment and validation of the hyperspectral images of GaoFen-5 (GF-5) and ZY1-02D were conducted separately using the HGF method, and the fusion effect was evaluated in three band intervals according to the spectral response of the ground class. First, HGF was used to fuse the GF-5 and GaoFen-1 (GF-1) images, and the fusion effect was evaluated both qualitatively and quantitatively. Second, the optimal fusion method was selected for the corresponding characteristic bands of the different surface elements. Finally, the hyperspectral image obtained by ZY1-02D and multispectral image of Sentinel-2B were used for validation to improve the accuracy and efficiency of satellite hyperspectral images in quantitative studies. The results show that for further studies on soil, vegetation, and water bodies, the best fusion methods in the 390–730, 730–1400, and 1400–2260 nm intervals are the GS, HGF, and HGF algorithms, respectively. Further analysis showed that the HGF or GS methods can be selected for quantitative studies on vegetation and water bodies and that the HGF method exhibits outstanding advantages for quantitative analysis of each soil element.

Joint sparse representation classification (JSRC) is a representative spectral–spatial classifier for hyperspectral images (HSIs). However, the JSRC is inappropriate for highly heterogeneous areas due to the spatial information being extracted from a fixed-sized neighborhood block, which is often unable to conform to the naturally irregular structure of land cover. To address this problem, a superpixel-based JSRC with nonlocal weighting, i.e., superpixel-based nonlocal weighted JSRC (SNLW-JSRC), is proposed in this paper. In SNLW-JSRC, the superpixel representation of an HSI is first constructed based on an entropy rate segmentation method. This strategy forms homogeneous neighborhoods with naturally irregular structures and alleviates the inclusion of pixels from different classes in the process of spatial information extraction. Afterwards, the superpixel-based nonlocal weighting (SNLW) scheme is built to weigh the superpixel based on its structural and spectral information. In this way, the weight of one specific neighboring pixel is determined by the local structural similarity between the neighboring pixel and the central test pixel. Then, the obtained local weights are used to generate the weighted mean data for each superpixel. Finally, JSRC is used to produce the superpixel-level classification. This speeds up the sparse representation and makes the spatial content more centralized and compact. To verify the proposed SNLW-JSRC method, we conducted experiments on four benchmark hyperspectral datasets, namely Indian Pines, Pavia University, Salinas, and DFC2013. The experimental results suggest that the SNLW-JSRC can achieve better classification results than the other four SRC-based algorithms and the classical support vector machine algorithm. Moreover, the SNLW-JSRC can also outperform the other SRC-based algorithms, even with a small number of training samples.

Copyright of Journal of Remote Sensing is the property of Editorial Office of Journal of Remote Sensing & Science Publishing Co. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In the field of spatial–spectral fusion, the variational model-based methods and the deep learning (DL)-based methods are state-of-the-art approaches. This paper presents a fusion method that combines the deep neural network with a variational model for the most common case of spatial–spectral fusion: panchromatic (PAN)/multispectral (MS) fusion. Specifically, a deep residual convolutional neural network (CNN) is first trained to learn the gradient features of the high spatial resolution multispectral image (HR-MS). The image observation variational models are then formulated to describe the relationships of the ideal fused image, the observed low spatial resolution multispectral image (LR-MS) image, and the gradient priors learned before. Then, fusion result can then be obtained by solving the fusion variational model. Both quantitative and visual assessments on high-quality images from various sources demonstrate that the proposed fusion method is superior to all the mainstream algorithms included in the comparison, in terms of overall fusion accuracy. [ABSTRACT FROM AUTHOR]

Unmanned Aerial Vehicles (UAV) have gained great importance for patrolling, exploration, and surveillance. In this study, we have estimated a route UAV to follow, using aerial road images. In the experimental setup, for estimation, test, and validation stages, anonymous aerial road videos have been exploited, meaning a special image database was not produced for this simulation approach. In the proposed study, road portion is initially detected. Two methods are utilized to help road detection, which are k-Nearest Neighbor and Hough transformation. To form a decision loop, both results are matched. If they match each other, they are fused using spatial and spectral schemes for the comparison purpose. Once road area is detected, the road type classification is realized by Fuzzy approach. The resultant image is utilized to estimate route, over which the UAV have to fly towards that direction. In the simulation stage, an anonymous video stream previously captured by UAV is experimented to assess the performance of the underlying system for different roads. According to the implementation results, the proposed algorithm has succeeded in finding all the trial roads in the given aerial images, and the proportion of all the estimated road-portion to actual road pixels for all the images is averagely calculated as %95.40. Eventually, it is shown that UAV has followed the correct route, which is estimated by proposed approach, over the specified road using assigned video frames, and also performances of spatial and spectral fusion results are compared. [ABSTRACT FROM AUTHOR]

Copyright of Transactions of Beijing Institute of Technology is the property of Beijing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

With the emergence of new remote sensing modalities, it becomes increasingly important to find novel algorithms for fusion and integration of different types of data for the purpose of improving performance of applications, such as target/anomaly detection or classification. Many popular techniques that deal with this problem are based on performing multiple classifications and fusing these individual results into one product. In this paper we provide a new approach, focused on creating joint representations of the multi-modal data, which then can be subject to analysis by state of the art classifiers. In the work presented in this paper we consider the problem of spatial-spectral fusion for hyperspectral imagery. Our approach involves machine learning techniques based on analysis of joint data-dependent graphs and the resulting data-dependent fusion operators and their representations. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel spatial and spectral fusion model (SASFM) that uses sparse matrix factorization to fuse remote sensing imagery with different spatial and spectral properties. By combining the spectral information from sensors with low spatial resolution (LSaR) but high spectral resolution (HSeR) (hereafter called HSeR sensors), with the spatial information from sensors with high spatial resolution (HSaR) but low spectral resolution (LSeR) (hereafter called HSaR sensors), the SASFM can generate synthetic remote sensing data with both HSaR and HSeR. Given two reasonable assumptions, the proposed model can integrate the LSaR and HSaR data via two stages. In the first stage, the model learns from the LSaR data a spectral dictionary containing pure signatures, and in the second stage, the desired HSaR and HSeR data are predicted using the learned spectral dictionary and the known HSaR data. The SASFM is tested with both simulated data and actual Landsat 7 Enhanced Thematic Mapper Plus (ETM+) and Terra Moderate Resolution Imaging Spectroradiometer (MODIS) acquisitions, and it is also compared to other representative algorithms. The experimental results demonstrate that the SASFM outperforms other algorithms in generating fused imagery with both the well-preserved spectral properties of MODIS and the spatial properties of ETM+. Generated imagery with simultaneous HSaR and HSeR opens new avenues for applications of MODIS and ETM+. [ABSTRACT FROM PUBLISHER]