Search

Pansharpening aims to integrate low-resolution multispectral (LRMS) images and panchromatic (PAN) images to obtain the high-resolution multispectral (HRMS) images. As PAN and LRMS images capture the same scene, they have some common information. Due to sensor characteristics’ differences, they also have unique information. Therefore, recent pansharpening methods separate the common and unique features to reduce redundancy. However, they are not explicitly equipped with specific modules to extract the common and unique features, which limits redundancy removal. To solve the problems, we propose a pansharpening method using a cross-attention-based module to specifically extract the common and unique feature maps from the MS and PAN images. In addition, mutual information (MI) maximization and minimization constraints are used to enforce the common and unique features, which can reduce the redundant information. Finally, these features are concatenated to generate the HRMS. Extensive experimental results on the QuickBird and Gaofen-2 datasets demonstrate that the proposed method outperforms other methods quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

This work proposes a simple yet effective method to adapt unsupervised convolutional neural networks (CNNs) from multispectral (MS) to hyperspectral (HS) pansharpening. Thus, it focuses on the fusion of a single high-resolution panchromatic (PAN) band with a low-resolution HS data cube. This is achieved by means of a decorrelation transform, following the principal component analysis (PCA) approach, which enables the compression of a significant portion of the HS image energy into a few bands. Afterward, a suitably adapted pansharpening network designed for four spectral bands is used to super-resolve only the principal components (PCs). Experiments demonstrate high performance in both quantitative and qualitative evaluations, favorably comparing against state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Pansharpening is a spatial-spectral fusion technique that fuses low-resolution multispectral (MS) images with high-resolution panchromatic (PAN) images to get high-resolution MS images which are rich in spectral and spatial information. Some pansharpening methods based on dynamic convolution were proposed to improve the adaptivity and generalization of the fusion network. However, these methods either only focus on local small regions or generate dynamic filters with a complex network. Besides, the dynamic filters in the existing methods only convolve with MS or PAN images, resulting in the extracted details or spectral features are inadequate. In this letter, we propose a dynamic large-small kernel convolutional network. To obtain small scale features, we propose a dual dynamic small kernel (DDSK) module which consists of dynamic spatial small filter (DASF) and dynamic spectral small filter (DESF). The multiscale dynamic large kernel convolution (MDLC) module is designed to expand the receptive field for obtaining large scale features. Considering the differences between PAN and MS images, DASF and spatial MDLC modules are presented to extract the details of PAN image. Similarly, DESF and spectral MDLC modules are proposed to obtain the spectral features of MS images. The proposed method is evaluated on GaoFen-2 and WorldView-3 datasets, and our method shows good performance. [ABSTRACT FROM AUTHOR]

The aim of pansharpening is to fuse panchromatic (PAN) images and the corresponding low-resolution multispectral (LRMS) images to generate high-resolution multispectral (HRMS) images. In recent years, convolutional neural network (CNN)-based methods have obtained excellent performance in this field. In order to break through the limitations of standard convolution, the dynamic convolution is also proposed for pansharpening. However, the existing dynamic convolution ignores the context along channel dimension and does not consider the mutual enhancement between spatial and spectral information. In this letter, we propose a novel cross-interaction kernel attention network (CIKANet). The proposed model consists of two branches to extract spectral and spatial information. In particular, spatial and spectral kernel attention modules that can dynamically scale the parameters of convolutional kernels are applied. To strengthen the interactions between branches and obtain complementary information, we adopt a cross kernel attention structure. A series of experiments conducted on the WV3 and QB datasets suggest that CIKANet outperforms other state-of-the-art (SOTA) models visually and quantitatively. [ABSTRACT FROM AUTHOR]

Recently, convolutional neural networks (CNNs) have been developed for remote sensing image fusion (RSIF). To obtain competitive fusion performance, network design becomes more complicated by stacking convolutional layers deeper and wider. However, problems still remain when applying the existing networks in practical applications. On the one hand, researchers focus on improving spatial resolution but ignore that the fused images will be used in subsequent interpretation applications, e.g., objection detection. On the other hand, RSIF involves different tasks with different image sources, e.g., pansharpening of the panchromatic and multispectral image (MSI), hypersharpening of the panchromatic and hyperspectral image (HSI), and so on. However, the existing networks only solve one of them, failing to be compatible with other tasks. To address the above problems, a convenient task-inspired multiscale nonlocal-attention network (MNAN) is proposed for RSIF. The proposed MNAN focuses more on enhancing the multiscale targets in the scene when improving the resolution of the fused image. In addition, the proposed network can be applied to both pansharpening and hypersharpening tasks without any modification. [ABSTRACT FROM AUTHOR]

Pansharpening is achieved by inferring spatial details derived from a PANchromatic (PAN) image into its corresponding expanded multispectral (MS) bands. In this letter, we propose to apply an adaptive superpixel-based injection scheme that modulates the PAN details through an optimization procedure. Optimal injection coefficients can be locally estimated by using the shuffled complex evolution developed in the University of Arizona (SCE-UA) algorithm over multiple local segments (i.e., superpixels) resulting from the simple linear iterative clustering (SLIC) method. The performance of the proposed approach is assessed using degraded and real data sets acquired from WorldView-3 and WorldView-4 satellites. Experimental results show the suitability of the proposed adaptive injection scheme compared with other state-of-the-art pansharpening methods in terms of spatial and spectral qualities. [ABSTRACT FROM AUTHOR]

Observation of the Earth using satellites mounted with optical sensors is an important application of remote sensing. Owing to physical constraints, multispectral (MS) sensors acquire images of lower spatial resolution than a single-band panchromatic (PAN) sensor that acquires images of the same scene. Pansharpening fuses the MS and PAN images to obtain an MS image with the same spatial resolution as the PAN image. In this letter, we propose to expand a method, initially developed for Sentinel-2 single-sensor sharpening, for pansharpening. The expanded method is based on solving a non-convex MS acquisition model using optimization methods based on cyclic decent and manifold optimization. The tuning parameters of the method are chosen using Bayesian optimization with reduced-scale evaluation. The proposed method is compared with a number of established pansharpening methods and is validated using both synthetic and real data sets. [ABSTRACT FROM AUTHOR]

Pansharpening techniques allow a detailed reproduction of the Earth surface by fusing a multispectral (MS) and a panchromatic (PAN) image acquired over the same area. Classical pansharpening methods consist in the extraction of the details from the PAN image and their subsequent injection into the MS image through a linear function. In this letter, we propose to apply a nonlinear injection procedure that implements the detail injection through a polynomial function. Optimal polynomial coefficients in the least squares sense can be easily obtained in the closed form, and the consequent pansharpening algorithm is shown to obtain superior performance with respect to the existing linear approaches, especially for MS bands with a reduced wavelength overlap with the PAN channel. [ABSTRACT FROM AUTHOR]

In this letter, we address the pansharpening problem, which focuses on constructing a high-resolution (HR) multispectral (MS) image from a low-resolution (LR) MS and an HR panchromatic (Pan) image. The accuracy of pansharpening method based on sparse representation (SR) mainly depends on the construction of dictionary and the learning of sparse coefficients, while the details injection (DI)-based pansharpening method sharpens the MS bands by adding the proper spatial details from Pan. The combination of SR and DI has been put forward as the pansharpening method based on SR of injected details (SR-D). However, limited to the patch-based manner, pansharpening with traditional SR model faces two disadvantages, i.e., limited ability in detail preservation and high sensitivity to misregistration. In this letter, we replace the traditional SR model with convolutional SR (CSR) as a global SR model in the SR-D method and propose a new pansharpening method called CSR of injected details (CSR-D) to overcome the above-mentioned two drawbacks. Experimental results on the IKONOS and WorldView2 data sets show that the proposed method can achieve remarkable spectral and spatial quality on both reduced scale and full scale. [ABSTRACT FROM AUTHOR]

In this letter, hypersharpening is analyzed in depth by investigating some weaknesses present in its formulation. It is shown that the key formula of the synthesized band variant can be simplified under certain circumstances. In addition, a novel fusion schema is proposed. As a result, the gain factor adopted to weight the injected detail is computed in a different way. This schema can be applied to fuse a wide range of hyperspectral and multispectral data. In this letter, its effectiveness is demonstrated by taking into account the characteristics of WorldView-3 data. [ABSTRACT FROM AUTHOR]

Pansharpening techniques aim at fusing a low-spatial resolution multispectral (MS) image with a higher spatial resolution panchromatic (PAN) image to produce an MS image at high spatial resolution. Despite significant progress in the field, spectral and spatial distortions might still compromise the quality of the results. We introduce a restoration strategy to mitigate artifacts of fused products. After applying the principal component analysis transform to a pansharpened image, the chromatic components are filtered conditionally to the geometry of PAN. The structural component is then replaced by the locally histogram-matched PAN for spatial enhancement. Experimental results illustrate the efficiency of the proposed restoration chain. [ABSTRACT FROM AUTHOR]

Pansharpening refers to the problem of fusing a multispectral (MS) image and a panchromatic image in order to get an MS image at a finer spatial resolution than the one of the original MS image. Due to the lack of a reference image, the assessment of the quality of a pansharpened product is a challenging task. Typical solutions are related to the reduced resolution assessment by exploiting Wald’s protocol or to indexes without reference. In this letter, an efficient approach based on the combination of the two above-mentioned solutions is proposed. The performance is assessed using real data acquired by sensors with very different features mounted on-board of the Pléiades, the WorldView-3, and the WorldView-4 satellite platforms. [ABSTRACT FROM AUTHOR]

In this letter, a new variational model in texture space for pansharpening is proposed to increase spatial information of multispectral image, while preserving spectral and spatial consistencies of pansharpened image. Geometric structure consistency between panchromatic image and pansharpened image is very important for preserving spatial characteristics, especially in borders, edges, and textured regions. G-norm can extract more pure and accurate texture, curvature, and oscillating details than do previous first-order and second-order-based methods, such as Gradient and Hessian operators. Therefore, we aim to use the G-space to maintain spatial information. Experimental results show that the proposed method has a better performance in terms of both spatial and spectral qualities; however, it is not efficient in terms of computation time. [ABSTRACT FROM AUTHOR]

Reconstruction of a sharpened multispectral (MS) image from its coarser measurements, namely the low spatial resolution MS and panchromatic (Pan) images, is a severely illposed inverse problem which requires the definition of an appropriate prior model. This letter incorporates an adaptive Markov random field (MRF)-based prior model into a Bayesian framework to recover the desired MS image. The proposed MRF-based prior model combines the high frequency details of the Pan image with the spectral relation between the bands of the MS image into a single energy function. Consequently, unlike most pansharpening methods, not directly injecting the spatial information of the Pan image into the fused product, the proposed method offers a fused product with minimum spectral distortion, along with perfectly enhanced spatial resolution. Visual and quantitative assessments of the fused products of the proposed method compared to those of some famous pansharpening methods prove the superiority of the proposed method. [ABSTRACT FROM PUBLISHER]

This letter discusses whether the expanded multispectral image, i.e., the original multispectral image upsampled to the panchromatic scale, can be used during the assessment of the quality of pansharpened multispectral images. By considering Wald’s protocol, the authors demonstrate that the adoption of the expanded image as the reference is erroneous and brings a quality assessment of the fused images that is misleading. In addition, some recommendations about the valid role of the expanded image are provided. The discussion is supported by a quantitative analysis and visual comparisons. [ABSTRACT FROM PUBLISHER]

WorldView 3 (WV-3) is the first commercially deployed super-spectral, very high-resolution (HR) satellite. However, the resolution of the short-wave infrared (SWIR) bands is much lower than that of the other bands. In this letter, we describe four different approaches, which are combinations of pansharpening and hypersharpening methods, to generate HR SWIR images. Since there are no ground truth HR SWIR images, we also propose a new picture quality predictor to assess hypersharpening performance, without the need for reference images. We describe extensive experiments using actual WV-3 images that demonstrate that some approaches can yield better performance than others, as measured by the proposed blind image quality assessment model of hypersharpened SWIR images. [ABSTRACT FROM PUBLISHER]

In the field of multispectral (MS) and panchromatic image fusion (pansharpening), the impressive effectiveness of deep neural networks has recently been employed to overcome the drawbacks of the traditional linear models and boost the fusion accuracy. However, the existing methods are mainly based on simple and flat networks with relatively shallow architectures, which severely limits their performance. In this letter, the concept of residual learning is introduced to form a very deep convolutional neural network to make the full use of the high nonlinearity of the deep learning models. Through both quantitative and visual assessments on a large number of high-quality MS images from various sources, it is confirmed that the proposed model is superior to all the mainstream algorithms included in the comparison, and achieves the highest spatial–spectral unified accuracy. [ABSTRACT FROM PUBLISHER]

The component substitution (CS) scheme is one of the most efficient models used by different image fusion algorithms when merging multispectral and panchromatic images, acquired with different spatial and spectral resolutions. In this letter, a new CS-based image fusion method is proposed to reduce color distortion. First, multispectral images are clustered into several classes using spectral and spatial features, and then linear regression with non-negative coefficients is used to calculate summation weights for each class of pixels. To consider mixed pixels not belonging to any distinct class, the proposed method employs the fuzzy $c$ -means algorithm. Qualitative and quantitative results are reported for two data sets, namely, Landsat-7 Enhanded Thematic Mapper Plus and QuickBird. Visual and statistical assessments show the validity of the proposed method. [ABSTRACT FROM PUBLISHER]

Classical pansharpening algorithms constitute a class of image fusion methods that have been widely investigated in the literature. They have been developed for combining a single- and a multichannel image (panchromatic (PAN) and multispectral (MS), respectively), but can be adapted to the sharpening of hyperspectral (HS) data, both through companion PAN and MS images. We focus in this letter on the HS/MS fusion, showing that the assignation of the MS channel to each HS band is a key step, and investigate several alternatives to make this choice. The assignment algorithms are tested in conjunction with both component substitution and multiresolution analysis pansharpening methods and assessed on images acquired by the Hyperion and ALI sensors. The numerical evaluation shows that the best results can be obtained by optimizing the spectral angle mapper metric confirming that classical methods represent a reliable basis for the development of novel sharpening algorithms. [ABSTRACT FROM PUBLISHER]

The application of sparse representation (SR) theory to the fusion of multispectral (MS) and panchromatic images is giving a large impulse to this topic, which is recast as a signal reconstruction problem from a reduced number of measurements. This letter presents an effective implementation of this technique, in which the application of SR is limited to the estimation of missing details that are injected in the available MS image to enhance its spatial features. We propose an algorithm exploiting the details self-similarity through the scales and compare it with classical and recent pansharpening methods, both at reduced and full resolution. Two different data sets, acquired by the WorldView-2 and IKONOS sensors, are employed for validation, achieving remarkable results in terms of spectral and spatial quality of the fused product. [ABSTRACT FROM PUBLISHER]

The synthesis of low-resolution panchromatic (Pan) image is a critical step of ratio enhancement (RE) and component substitution (CS) pansharpening methods. The two types of methods assume a linear relation between Pan and multispectral (MS) images. However, due to the nonlinear spectral response of satellite sensors, the qualified low-resolution Pan image cannot be well approximated by a weighted summation of MS bands. Therefore, in some local areas, significant gray value difference exists between a synthetic Pan image and a high-resolution Pan image. To tackle this problem, the pixels of Pan and MS images are divided into several classes by $k$-means algorithm, and then multiple regression is used to calculate summation weights on each group of pixels. Experimental results demonstrate that the proposed technique can provide significant improvements on reducing color distortion. [ABSTRACT FROM PUBLISHER]

An efficient pansharpening method should inject the missing geometric information to the multispectral (MS) image while preserving its radiometric information. Widely used additive wavelet transform-based pansharpening methods extract the missing high-frequency information by decomposing the panchromatic (PAN) image and adding the detail layers to the low-resolution MS (LRM) image. However, this approach causes a redundant detail injection, leading to artifacts in the fusion result. In this letter, we propose to decompose the high-resolution-PAN image using an edge-preserving decomposition which will decrease the amount of redundant high-frequency injection. The missing high-frequency information of the LRM image is obtained by the decomposition of the PAN image using a multiscale bilateral filter. The spatial and range parameters of the bilateral filter are optimized so as to enhance spatial and spectral metrics. The fusion results are compared with the widely used additive wavelet luminance proportional (AWLP) and recently proposed improved AWLP fusion methods. The resulting images as well as evaluation metrics demonstrate that the proposed injection approach has better performance. [ABSTRACT FROM PUBLISHER]

The pansharpening process has the purpose of building a high-resolution multispectral image by fusing low spatial resolution multispectral and high-resolution panchromatic observations. A very credited method to pursue this goal relies upon the injection of details extracted from the panchromatic image into an upsampled version of the low-resolution multispectral image. In this letter, we compare two different injection methodologies and motivate the superiority of contrast-based methods both by physical consideration and by numerical tests carried out on remotely sensed data acquired by IKONOS and Quickbird sensors. [ABSTRACT FROM PUBLISHER]

In this letter, we present a new method for the pansharpening of multispectral satellite imagery. Pansharpening is the process of synthesizing a high spatial resolution multispectral image from a low spatial resolution multispectral image and a high-resolution panchromatic (PAN) image. The method uses total variation to regularize an ill-posed problem dictated by a widely used explicit image formation model. This model is based on the assumptions that a linear combination of the bands of the pansharpened image gives the PAN image and that a decimation of the pansharpened image gives the original multispectral image. Experimental results are based on two real datasets and the quantitative quality of the pansharpened images is evaluated using a number of spatial and spectral metrics, some of which have been recently proposed and do not need a reference image. The proposed method compares favorably to other well-known methods for pansharpening and produces images of excellent spatial and spectral quality. [ABSTRACT FROM PUBLISHER]

State-of-the-art pansharpening methods generally inject the spatial details extracted from the panchromatic (Pan) image into the multispectral (MS) images by considering different injection models. The fusion performances severely rely on the accuracy of the modeling and the estimation of model parameters. In this letter, we propose an optimized approach to avoid explicitly modeling the detail injection process. The solution employs the gradient field of the Pan image for spatial enhancement. The low-pass (LP) version of the fused bands are constrained to be the most similar to the original MS bands to preserve the spectral characteristics. We use the local correlation coefficients between the MS band and the LP version of the Pan image to adjust the two sources of information based on a simple observation, and it is further optimized by considering the overall quality index Q4. Experimental results demonstrate that the proposed method outperforms the state-of-the-art multiresolution analysis-based methods. [ABSTRACT FROM PUBLISHER]

In this letter, we propose a novel pansharpening method which enhances the spatial resolution of low-spatial-resolution multispectral (MS) images through proportional detail injection (PDI) based on the generalized relative spectral response (GRSR). In the algorithm, the spatial details are extracted as the difference between the panchromatic (PAN) image and the resampled low-resolution PAN image. Then, the spatial details are further modulated and injected into each band of the resampled MS image according to the proportion of the intensity of the low-frequency component of each band to the offset-removed synthetic intensity component obtained according to the GRSR. Both quantitative and qualitative evaluations demonstrate the superiority of the proposed method in terms of spatial enhancement, spectral fidelity, and computational complexity. [ABSTRACT FROM PUBLISHER]