Search

Urban trees support vital ecological functions and help with the mitigation of and adaption to climate change. Yet, their monitoring and management require significant public resources. remote sensing could facilitate these tasks. Recent hyperspectral satellite programs such as PRISMA have enabled more advanced remote sensing applications, such as species classification. However, PRISMA data's spatial resolution (30 m) could limit its utility in urban areas. Improving hyperspectral data resolution with pansharpening using the PRISMA coregistered panchromatic band (spatial resolution of 5 m) could solve this problem. This study addresses the need to improve hyperspectral data resolution and tests the pansharpening method by classifying exemplative urban tree species in Naples (Italy) using a convolutional neural network and a ground truths dataset, with the aim of comparing results from the original 30 m data to data refined to a 5 m resolution. An evaluation of accuracy metrics shows that pansharpening improves classification quality in dense urban areas with complex topography. In fact, pansharpened data led to significantly higher accuracy for all the examined species. Specifically, the Pinus pinea and Tilia x europaea classes showed an increase of 10% to 20% in their F1 scores. Pansharpening is seen as a practical solution to enhance PRISMA data usability in urban environments. [ABSTRACT FROM AUTHOR]

Pansharpening refers to enhancing the spatial resolution of multispectral images through panchromatic images while preserving their spectral features. However, existing traditional methods or deep learning methods always have certain distortions in the spatial or spectral dimensions. This paper proposes a remote sensing spatial–spectral fusion method based on a multi-scale dual-stream convolutional neural network, which includes feature extraction, feature fusion, and image reconstruction modules for each scale. In terms of feature fusion, we propose a multi cascade module to better fuse image features. We also design a new loss function aim at enhancing the high degree of consistency between fused images and reference images in terms of spatial details and spectral information. To validate its effectiveness, we conduct thorough experimental analyses on two widely used remote sensing datasets: GeoEye-1 and Ikonos. Compared with the nine leading pansharpening techniques, the proposed method demonstrates superior performance in multiple key evaluation metrics. [ABSTRACT FROM AUTHOR]

In the field of remote sensing image processing, pansharpening technology stands as a critical advancement. This technology aims to enhance multispectral images that possess low resolution by integrating them with high-spatial-resolution panchromatic images, ultimately producing multispectral images with high resolution that are abundant in both spatial and spectral details. Thus, there remains potential for improving the quality of both the spectral and spatial domains of the fused images based on deep-learning-based pansharpening methods. This work proposes a new method for the task of pansharpening: the Multi-Stage Progressive Pansharpening Network with Detail Injection with Redundancy Reduction Mechanism (MSPPN-DIRRM). This network is divided into three levels, each of which is optimized for the extraction of spectral and spatial data at different scales. Particular spectral feature and spatial detail extraction modules are used at each stage. Moreover, a new image reconstruction module named the DRRM is introduced in this work; it eliminates both spatial and channel redundancy and improves the fusion quality. The effectiveness of the proposed model is further supported by experimental results using both simulated data and real data from the QuickBird, GaoFen1, and WorldView2 satellites; these results show that the proposed model outperforms deep-learning-based methods in both visual and quantitative assessments. Among various evaluation metrics, performance improves by 0.92–18.7% compared to the latest methods. [ABSTRACT FROM AUTHOR]

In recent years, deep learning techniques for pansharpening multiresolution images have gained increasing interest. Due to the lack of ground truth data, most deep learning solutions rely on synthetic reduced-resolution data for supervised training. This approach has limitations due to the statistical mismatch between real full-resolution and synthetic reduced-resolution data, which affects the models' generalization capacity. Consequently, there has been a shift towards unsupervised learning frameworks for pansharpening deep learning-based techniques. Unsupervised schemes require defining sophisticated loss functions with at least two components: one for spectral quality, ensuring consistency between the pansharpened image and the input multispectral component, and another for spatial quality, ensuring consistency between the output and the panchromatic input. Despite promising results, there has been limited investigation into the interaction and balance of these loss terms to ensure stability and accuracy. This work explores how unsupervised spatial and spectral consistency losses can be reliably combined preserving the outocome quality. By examining these interactions, we propose a general rule for balancing the two loss components to enhance the stability and performance of unsupervised pansharpening models. Experiments on three state-of-the-art algorithms using WorldView-3 images demonstrate that methods trained with the proposed framework achieve good performance in terms of visual quality and numerical indexes. [ABSTRACT FROM AUTHOR]

Pansharpening refers to the process of fusing multispectral (MS) images with panchromatic (PAN) images to obtain high-resolution multispectral (HRMS) images. However, due to the low correlation and similarity between MS and PAN images, as well as inaccuracies in spatial information injection, HRMS images often suffer from significant spectral and spatial distortions. To address these issues, a pansharpening method based on multimodal texture correction and adaptive edge detail fusion is proposed in this paper. To obtain a texture-corrected ( T C ) image that is highly correlated and similar to the MS image, the target-adaptive CNN-based pansharpening (A-PNN) method is introduced. By constructing a multimodal texture correction model, intensity, gradient, and A-PNN-based deep plug-and-play correction constraints are established between the T C and source images. Additionally, an adaptive degradation filter algorithm is proposed to ensure the accuracy of these constraints. Since the T C image obtained can effectively replace the PAN image and considering that the MS image contains valuable spatial information, an adaptive edge detail fusion algorithm is also proposed. This algorithm adaptively extracts detailed information from the T C and MS images to apply edge protection. Given the limited spatial information in the MS image, its spatial information is proportionally enhanced before the adaptive fusion. The fused spatial information is then injected into the upsampled multispectral (UPMS) image to produce the final HRMS image. Extensive experimental results demonstrated that compared with other methods, the proposed algorithm achieved superior results in terms of both subjective visual effects and objective evaluation metrics. [ABSTRACT FROM AUTHOR]

Pansharpening, a pivotal task in remote sensing, involves integrating low-resolution multispectral images with high-resolution panchromatic images to synthesize an image that is both high-resolution and retains multispectral information. These pansharpened images enhance precision in land cover classification, change detection, and environmental monitoring within remote sensing data analysis. While deep learning techniques have shown significant success in pansharpening, existing methods often face limitations in their evaluation, focusing on restricted satellite data sources, single scene types, and low-resolution images. This paper addresses this gap by introducing PanBench, a high-resolution multi-scene dataset containing all mainstream satellites and comprising 5898 pairs of samples. Each pair includes a four-channel (RGB + near-infrared) multispectral image of 256 × 256 pixels and a mono-channel panchromatic image of 1024 × 1024 pixels. To avoid irreversible loss of spectral information and achieve a high-fidelity synthesis, we propose a Cascaded Multiscale Fusion Network (CMFNet) for pansharpening. Multispectral images are progressively upsampled while panchromatic images are downsampled. Corresponding multispectral features and panchromatic features at the same scale are then fused in a cascaded manner to obtain more robust features. Extensive experiments validate the effectiveness of CMFNet. [ABSTRACT FROM AUTHOR]

The objective of the pansharpening task is to integrate multispectral (MS) images with low spatial resolution (LR) and to integrate panchromatic (PAN) images with high spatial resolution (HR) to generate HRMS images. Recently, deep learning-based pansharpening methods have been widely studied. However, traditional deep learning methods lack transparency while deep unrolling methods have limited performance when using one implicit prior for HRMS images. To address this issue, we incorporate one implicit prior with a semi-implicit prior and propose a double prior deep unrolling network (DPDU-Net) for pansharpening. Specifically, we first formulate the objective function based on observation models of PAN and LRMS images and two priors of an HRMS image. In addition to the implicit prior in the image domain, we enforce the sparsity of the HRMS image in a certain multi-scale implicit space; thereby, the feature map can obtain better sparse representation ability. We optimize the proposed objective function via alternating iteration. Then, the iterative process is unrolled into an elaborate network, with each iteration corresponding to a stage of the network. We conduct both reduced-resolution and full-resolution experiments on two satellite datasets. Both visual comparisons and metric-based evaluations consistently demonstrate the superiority of the proposed DPDU-Net. [ABSTRACT FROM AUTHOR]

Hyperspectral pansharpening is crucial for the improvement of the usability of images in various applications. However, it remains underexplored due to a scarcity of data. The primary goal of pansharpening is to enhance the spatial resolution of hyperspectral images by reconstructing missing spectral information without compromising consistency with the original data. This paper addresses the data gap by presenting a new hyperspectral dataset specifically designed for pansharpening and the evaluation of several deep learning strategies using this dataset. The new dataset has two crucial features that make it invaluable for deep learning hyperspectral pansharpening research. (1) It presents the highest cardinality of images in the state of the art, making it the first statistically relevant dataset for hyperspectral pansharpening evaluation, and (2) it includes a wide variety of scenes, ensuring robust generalization capabilities for various approaches. The data, collected by the ASI PRISMA satellite, cover about 262,200 km2 and their heterogeneity is ensured by a random sampling of the Earth's surface. The analysis of the deep learning methods consists in the adaptation of these approaches to the PRISMA hyperspectral data and the quantitative and qualitative evaluation of their performance in this new scenario. The investigation included two settings: Reduced Resolution (RR) to evaluate the techniques in a controlled environment and Full Resolution (FR) for a real-world evaluation. In addition, for the sake of completeness, we have also included machine-learning-free approaches in both scenarios. Our comprehensive analysis reveals that data-driven neural network methods significantly outperform traditional approaches, demonstrating a superior adaptability and performance in hyperspectral pansharpening under both RR and FR protocols. [ABSTRACT FROM AUTHOR]

Image preprocessing and fusion are commonly used for enhancing remote-sensing images, but the resulting images often lack useful spatial features. As the majority of research on image fusion has concentrated on the satellite domain, the image-fusion task for Unmanned Aerial Vehicle (UAV) images has received minimal attention. This study investigated an image-improvement strategy by integrating image preprocessing and fusion tasks for UAV images. The goal is to improve spatial details and avoid color distortion in fused images. Techniques such as image denoising, sharpening, and Contrast Limited Adaptive Histogram Equalization (CLAHE) were used in the preprocessing step. The unsharp mask algorithm was used for image sharpening. Wiener and total variation denoising methods were used for image denoising. The image-fusion process was conducted in two steps: (1) fusing the spectral bands into one multispectral image and (2) pansharpening the panchromatic and multispectral images using the PanColorGAN model. The effectiveness of the proposed approach was evaluated using quantitative and qualitative assessment techniques, including no-reference image quality assessment (NR-IQA) metrics. In this experiment, the unsharp mask algorithm noticeably improved the spatial details of the pansharpened images. No preprocessing algorithm dramatically improved the color quality of the enhanced images. The proposed fusion approach improved the images without importing unnecessary blurring and color distortion issues. [ABSTRACT FROM AUTHOR]

Joint super-resolution and pansharpening (JSP) brings new insight into the spatial improvement of multispectral images. How to efficiently balance the spatial and spectral qualities in JSP is important for deep learning-based approaches. To address this problem, we propose a unified interpretable deep network for JSP, named UIJSP-Net. First, we formulate the JSP problem as an optimization problem in a specially designed physical model based on the relationship among the JSP result, the multispectral image, and the panchromatic image. In particular, two deep priors are utilized to describe latent distributions of different variables, which can improve the accuracy of the physical model. Furthermore, we adopt the alternating direction method of multipliers to solve the above optimization problem, where a series of iterative steps are generated. Finally, we design UIJSP-Net by unfolding these iterative steps into multiple corresponding stages in a unified network. Because UIJSP-Net has clear physical meanings, the spatial resolution of multispectral images can be efficiently improved while the spectral information can be kept as well. Extensive experimental results are carried out on both simulated and real datasets to demonstrate the superiority of UIJSP-Net over other state-of-the-art methods from qualitative and quantitative aspects. [ABSTRACT FROM AUTHOR]

The Sustainable Development Science Satellite 1 (SDGSAT-1) satellite, launched in November 2021, is dedicated to providing data detailing the "traces of human activities" for the implementation of the United Union's 2030 Agenda for Sustainable Development and global scientific research. The glimmer imager (GI) that is equipped on SDGSAT-1 can provide nighttime light (NL) data with a 10 m panchromatic (PAN) band and red, green, and blue (RGB) bands of 40 m resolution, which can be used for a wide range of applications, such as in urban expansion, population studies of cities, and economics of cities, as well as nighttime aerosol thickness monitoring. The 10 m PAN band can be fused with the 40 m RGB bands to obtain a 10 m RGB NL image, which can be used to identify the intensity and type of night lights and the spatial distribution of road networks and to improve the monitoring accuracy of sustainable development goal (SDG) indicators related to city developments. Existing remote sensing image fusion algorithms are mainly developed for daytime optical remote sensing images. Compared with daytime optical remote sensing images, NL images are characterized by a large amount of dark (low-value) pixels and high background noises. To investigate whether daytime optical image fusion algorithms are suitable for the fusion of GI NL images and which image fusion algorithms are the best choice for GI images, this study conducted a comprehensive evaluation of thirteen state-of-the-art pansharpening algorithms in terms of quantitative indicators and visual inspection using four GI NL datasets. The results showed that PanNet, GLP_HPM, GSA, and HR outperformed the other methods and provided stable performances among the four datasets. Specifically, PanNet offered UIQI values ranging from 0.907 to 0.952 for the four datasets, whereas GSA, HR, and GLP_HPM provided UIQI values ranging from 0.770 to 0.856. The three methods based on convolutional neural networks achieved more robust and better visual effects than the methods using multiresolution analysis at the original scale. According to the experimental results, PanNet shows great potential in the fusion of SDGSAT-1 GI imagery due to its robust performance and relatively short training time. The quality metrics generated at the degraded scale were highly consistent with visual inspection, but those used at the original scale were inconsistent with visual inspection. [ABSTRACT FROM AUTHOR]

Pansharpening is a technique used in remote sensing to combine high-resolution panchromatic (PAN) images with lower resolution multispectral (MS) images to generate high-resolution multispectral images while preserving spectral characteristics. Recently, convolutional neural networks (CNNs) have been the mainstream in pansharpening by extracting the deep features of PAN and MS images and fusing these abstract features to reconstruct high-resolution details. However, they are limited by the short-range contextual dependencies of convolution operations. Although transformer models can alleviate this problem, they still suffer from weak capability in reconstructing high-resolution detailed information from global representations. To this end, a novel Swin-transformer-based pansharpening model named SwinPAN is proposed. Specifically, a detail reconstruction network (DRNet) is developed in an image difference and residual learning framework to reconstruct the high-resolution detailed information from the original images. DRNet is developed based on the Swin Transformer with a dynamic high-pass preservation module with adaptive convolution kernels. The experimental results on three remote sensing datasets with different sensors demonstrate that the proposed approach performs better than state-of-the-art networks through qualitative and quantitative analysis. Specifically, the generated pansharpening results contain finer spatial details and richer spectral information than other methods. [ABSTRACT FROM AUTHOR]

This paper proposes a pansharpening model in order to obtain remote-sensing images with high spatial resolution and high spectral resolution. Based on a generic component substitution (CS) fusion framework, the model utilizes the difference between the high-frequency component of the panchromatic (PAN) image and the high-frequency component of the luminance (L) image to express the missing spatial detail information of the ideal high-resolution multispectral (HRMS) image. A rolling guidance filter (RGF) is used in this framework to achieve the effective extraction of high-frequency information from remote-sensing images while reducing the spectral distortion of subsequent operations. The modulation transfer function (MTF) values of the sensor are also applied to the selection of adaptive weighting coefficients to further improve the spectral fidelity of the fused images. At the same time, the choice of suitable interpolation and gain coefficients improves the generalizability of the model while reducing spectral and spatial distortions. Finally, the use of a guided filter (GF) also greatly improves the quality of the fused image. The experimental results show that the model can effectively improve the spatial resolution for foreign objects at the perimeter of high-speed railways, while also ensuring the color fidelity of foreign objects such as colored steel tiles. [ABSTRACT FROM AUTHOR]

Pansharpening refers to the fusion of a panchromatic (PAN) and a multispectral (MS) image aimed at generating a high-quality outcome over the same area. This particular image fusion problem has been widely studied, but until recently, it has been challenging to balance the spatial and spectral fidelity in fused images. The spectral distortion is widespread in the component substitution-based approaches due to the variation in the intensity distribution of spatial components. We lightened the idea using the total variation optimization to improve upon a novel GIHS-TV framework for pansharpening. The framework drew the high spatial fidelity from the GIHS scheme and implemented it with a simpler variational expression. An improved L1-TV constraint to the new spatial–spectral information was introduced to the GIHS-TV framework, along with its fast implementation. The objective function was solved by the Iteratively Reweighted Norm (IRN) method. The experimental results on the "PAirMax" dataset clearly indicated that GIHS-TV could effectively reduce the spectral distortion in the process of component substitution. Our method has achieved excellent results in visual effects and evaluation metrics. [ABSTRACT FROM AUTHOR]

Pansharpening refers to the use of a panchromatic image to improve the spatial resolution of a multi-spectral image while preserving spectral signatures. However, existing pansharpening methods are still unsatisfactory at balancing the trade-off between spatial enhancement and spectral fidelity. In this paper, a multi-scale and multi-stream fusion network (named MMFN) that leverages the multi-scale information of the source images is proposed. The proposed architecture is simple, yet effective, and can fully extract various spatial/spectral features at different levels. A multi-stage reconstruction loss was adopted to recover the pansharpened images in each multi-stream fusion block, which facilitates and stabilizes the training process. The qualitative and quantitative assessment on three real remote sensing datasets (i.e., QuickBird, Pléiades, and WorldView-2) demonstrates that the proposed approach outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In the last few years, there has been a renewed interest in data fusion techniques, and, in particular, in pansharpening due to a paradigm shift from model-based to data-driven approaches, supported by the recent advances in deep learning. Although a plethora of convolutional neural networks (CNN) for pansharpening have been devised, some fundamental issues still wait for answers. Among these, cross-scale and cross-datasets generalization capabilities are probably the most urgent ones since most of the current networks are trained at a different scale (reduced-resolution), and, in general, they are well-fitted on some datasets but fail on others. A recent attempt to address both these issues leverages on a target-adaptive inference scheme operating with a suitable full-resolution loss. On the downside, such an approach pays an additional computational overhead due to the adaptation phase. In this work, we propose a variant of this method with an effective target-adaptation scheme that allows for the reduction in inference time by a factor of ten, on average, without accuracy loss. A wide set of experiments carried out on three different datasets, GeoEye-1, WorldView-2 and WorldView-3, prove the computational gain obtained while keeping top accuracy scores compared to state-of-the-art methods, both model-based and deep-learning ones. The generality of the proposed solution has also been validated, applying the new adaptation framework to different CNN models. [ABSTRACT FROM AUTHOR]

The fusion of a high-spatial-resolution panchromatic (PAN) image and a corresponding low-resolution multispectral (MS) image can yield a high-resolution multispectral (HRMS) image, which is also known as pansharpening. Most previous methods based on convolutional neural networks (CNNs) have achieved remarkable results. However, information of different scales has not been fully mined and utilized, and still produces spectral and spatial distortion. In this work, we propose a multilevel parallel feature injection network that contains three scale levels and two parallel branches. In the feature extraction branch, a multi-scale perception dynamic convolution dense block is proposed to adaptively extract the spatial and spectral information. Then, the sufficient multilevel features are injected into the image reconstruction branch, and an attention fusion module based on the spectral dimension is designed in order to fuse shallow contextual features and deep semantic features. In the image reconstruction branch, cascaded transformer blocks are employed to capture the similarities among the spectral bands of the MS image. Extensive experiments are conducted on the QuickBird and WorldView-3 datasets to demonstrate that MPFINet achieves significant improvement over several state-of-the-art methods on both spatial and spectral quality assessments. [ABSTRACT FROM AUTHOR]

Advancements in remote sensing have led to the development of Geographic Object-Based Image Analysis (GEOBIA). This method of information extraction focuses on segregating correlated pixels into groups for easier classification. This is of excellent use in analyzing very-high-resolution (VHR) data. The application of GEOBIA for glacier surface mapping, however, necessitates multiple scales of segmentation and input of supportive ancillary data. The mapping of glacier surface facies presents a unique problem to GEOBIA on account of its separable but closely matching spectral characteristics and often disheveled surface. Debris cover can induce challenges and requires additions of slope, temperature, and short-wave infrared data as supplements to enable efficient mapping. Moreover, as the influence of atmospheric corrections and image sharpening can derive variations in the apparent surface reflectance, a robust analysis of the effects of these processing routines in a GEOBIA environment is lacking. The current study aims to investigate the impact of three atmospheric corrections, Dark Object Subtraction (DOS), Quick Atmospheric Correction (QUAC), and Fast Line-of-Sight Atmospheric Analysis of Hypercubes (FLAASH), and two pansharpening methods, viz., Gram–Schmidt (GS) and Hyperspherical Color Sharpening (HCS), on the classification of surface facies using GEOBIA. This analysis is performed on VHR WorldView-2 imagery of selected glaciers in Ny-Ålesund, Svalbard, and Chandra–Bhaga basin, Himalaya. The image subsets are segmented using multiresolution segmentation with constant parameters. Three rule sets are defined: rule set 1 utilizes only spectral information, rule set 2 contains only spatial and contextual features, and rule set 3 combines both spatial and spectral attributes. Rule set 3 performs the best across all processing schemes with the highest overall accuracy, followed by rule set 1 and lastly rule set 2. This trend is observed for every image subset. Among the atmospheric corrections, DOS displays consistent performance and is the most reliable, followed by QUAC and FLAASH. Pansharpening improved overall accuracy and GS performed better than HCS. The study reports robust segmentation parameters that may be transferable to other VHR-based glacier surface facies mapping applications. The rule sets are adjusted across the processing schemes to adjust to the change in spectral characteristics introduced by the varying routines. The results indicate that GEOBIA for glacier surface facies mapping may be less prone to the differences in spectral signatures introduced by different atmospheric corrections but may respond well to increasing spatial resolution. The study highlighted the role of spatial attributes for mapping fine features, and in combination with appropriate spectral features may enhance thematic classification. [ABSTRACT FROM AUTHOR]

High-resolution (HR) multispectral (MS) images contain sharper detail and structure compared to the ground truth high-resolution hyperspectral (HS) images. In this paper, we propose a novel supervised learning method, which considers pansharpening as the spectral super-resolution of high-resolution multispectral images and generates high-resolution hyperspectral images. The proposed method learns the spectral mapping between high-resolution multispectral images and the ground truth high-resolution hyperspectral images. To consider the spectral correlation between bands, we build a three-dimensional (3D) convolution neural network (CNN). The network consists of three parts using an encoder–decoder framework: spatial/spectral feature extraction from high-resolution multispectral images/low-resolution (LR) hyperspectral images, feature transform, and image reconstruction to generate the results. In the image reconstruction network, we design the spatial–spectral fusion (SSF) blocks to reuse the extracted spatial and spectral features in the reconstructed feature layer. Then, we develop the discrepancy-based deep hybrid gradient (DDHG) losses with the spatial–spectral gradient (SSG) loss and deep gradient transfer (DGT) loss. The spatial–spectral gradient loss and deep gradient transfer loss are developed to preserve the spatial and spectral gradients from the ground truth high-resolution hyperspectral images and high-resolution multispectral images. To overcome the spectral and spatial discrepancy between two images, we design a spectral downsampling (SD) network and a gradient consistency estimation (GCE) network for hybrid gradient losses. In the experiments, it is seen that the proposed method outperforms the state-of-the-art methods in the subjective and objective experiments in terms of the structure and spectral preservation of high-resolution hyperspectral images. [ABSTRACT FROM AUTHOR]

Recently, with the rapid development of deep learning (DL), an increasing number of DL-based methods are applied in pansharpening. Benefiting from the powerful feature extraction capability of deep learning, DL-based methods have achieved state-of-the-art performance in pansharpening. However, most DL-based methods simply fuse multi-spectral (MS) images and panchromatic (PAN) images by concatenating, which can not make full use of the spectral information and spatial information of MS and PAN images, respectively. To address this issue, we propose a spectral-spatial interaction Network (SSIN) for pansharpening. Different from previous works, we extract the features of PAN and MS, respectively, and then interact them repetitively to incorporate spectral and spatial information progressively. In order to enhance the spectral-spatial information fusion, we further propose spectral-spatial attention (SSA) module to yield a more effective spatial-spectral information transfer in the network. Extensive experiments on QuickBird, WorldView-4, and WorldView-2 images demonstrate that our SSIN significantly outperforms other methods in terms of both objective assessment and visual quality. [ABSTRACT FROM AUTHOR]

Pansharpening based on deep learning (DL) has shown great advantages. Most convolutional neural network (CNN)-based methods focus on obtaining local features from multispectral (MS) and panchromatic (PAN) images, but ignore the nonlocal dependence on images. Therefore, Transformer-based methods are introduced to obtain long-range information on images. However, the representational capabilities of features extracted by CNN or Transformer alone are weak. To solve this problem, a local and nonlocal feature interaction network (LNFIN) is proposed in this paper for pansharpening. It comprises Transformer and CNN branches. Furthermore, a feature interaction module (FIM) is proposed to fuse different features and return to the two branches to enhance the representational capability of features. Specifically, a CNN branch consisting of multiscale dense modules (MDMs) is proposed for acquiring local features of the image, and a Transformer branch consisting of pansharpening Transformer modules (PTMs) is introduced for acquiring nonlocal features of the image. In addition, inspired by the PTM, a shift pansharpening Transformer module (SPTM) is proposed for the learning of texture features to further enhance the spatial representation of features. The LNFIN outperforms the state-of-the-art method experimentally on three datasets. [ABSTRACT FROM AUTHOR]

Due to the inability of convolutional neural networks to effectively obtain long-range information, a transformer was recently introduced into the field of pansharpening to obtain global dependencies. However, a transformer does not pay enough attention to the information of channel dimensions. To solve this problem, a local-global-based high-resolution spatial-spectral representation network (LG-HSSRN) is proposed to fully fuse local and global spatial-spectral information at different scales. In this paper, a multi-scale feature fusion (MSFF) architecture is designed to obtain the scale information of remote sensing images. Meanwhile, in order to learn spatial texture information and spectral information effectively, a local-global feature extraction (LGFE) module is proposed to capture the local and global dependencies in the source images from a spatial-spectral perspective. In addition, a multi-scale contextual aggregation (MSCA) module is proposed to weave hierarchical information with high representational power. The results of three satellite datasets show that the proposed method exhibits superior performance in terms of both spatial and spectral preservation compared to other methods. [ABSTRACT FROM AUTHOR]

Due to the discrepancy in spatial structure between multispectral (MS) and panchromatic (PAN) images, the general fusion scheme will lead to image error in the fused result. To solve this issue, a differential strategy-based multi-level dense network is proposed, and it regards the image pairs at different scales as the input of the network at different levels and is able to map the spatial information in PAN images to each band of MS images well by learning the differential information of different levels, which effectively solves the scale effect of remote sensing images. An improved dense network with the same hierarchical structure is used to obtain richer spatial features to enhance the spatial information of the fused result. Meanwhile, a hybrid loss strategy is used to constrain the network at different levels for obtaining better results. Qualitative and quantitative analyses show that the result has a uniform spectral distribution, a complete spatial structure, and optimal evaluation criteria, which fully demonstrate the superior performance of the proposed method. [ABSTRACT FROM AUTHOR]

Glacier surface facies are valuable indicators of changes experienced by a glacial system. The interplay of accumulation and ablation facies, followed by intermixing with dust and debris, as well as the local climate, all induce observable and mappable changes on the supraglacial terrain. In the absence or lag of continuous field monitoring, remote sensing observations become vital for maintaining a constant supply of measurable data. However, remote satellite observations suffer from atmospheric effects, resolution disparity, and use of a multitude of mapping methods. Efficient image-processing routines are, hence, necessary to prepare and test the derivable data for mapping applications. The existing literature provides an application-centric view for selection of image processing schemes. This can create confusion, as it is not clear which method of atmospheric correction would be ideal for retrieving facies spectral reflectance, nor are the effects of pansharpening examined on facies. Moreover, with a variety of supervised classifiers and target detection methods now available, it is prudent to test the impact of variations in processing schemes on the resultant thematic classifications. In this context, the current study set its experimental goals. Using very-high-resolution (VHR) WorldView-2 data, we aimed to test the effects of three common atmospheric correction methods, viz. Dark Object Subtraction (DOS), Quick Atmospheric Correction (QUAC), and Fast Line-of-Sight Atmospheric Analysis of Hypercubes (FLAASH); and two pansharpening methods, viz. Gram–Schmidt (GS) and Hyperspherical Color Sharpening (HCS), on thematic classification of facies using 12 supervised classifiers. The conventional classifiers included: Mahalanobis Distance (MHD), Maximum Likelihood (MXL), Minimum Distance to Mean (MD), Spectral Angle Mapper (SAM), and Winner Takes All (WTA). The advanced/target detection classifiers consisted of: Adaptive Coherence Estimator (ACE), Constrained Energy Minimization (CEM), Matched Filtering (MF), Mixture-Tuned Matched Filtering (MTMF), Mixture-Tuned Target-Constrained Interference-Minimized Filter (MTTCIMF), Orthogonal Space Projection (OSP), and Target-Constrained Interference-Minimized Filter (TCIMF). This experiment was performed on glaciers at two test sites, Ny-Ålesund, Svalbard, Norway; and Chandra–Bhaga basin, Himalaya, India. The overall performance suggested that the FLAASH correction delivered realistic reflectance spectra, while DOS delivered the least realistic. Spectra derived from HCS sharpened subsets seemed to match the average reflectance trends, whereas GS reduced the overall reflectance. WTA classification of the DOS subsets achieved the highest overall accuracy (0.81). MTTCIMF classification of the FLAASH subsets yielded the lowest overall accuracy of 0.01. However, FLAASH consistently provided better performance (less variable and generally accurate) than DOS and QUAC, making it the more reliable and hence recommended algorithm. While HCS-pansharpened classification achieved a lower error rate (0.71) in comparison to GS pansharpening (0.76), neither significantly improved accuracy nor efficiency. The Ny-Ålesund glacier facies were best classified using MXL (error rate = 0.49) and WTA classifiers (error rate = 0.53), whereas the Himalayan glacier facies were best classified using MD (error rate = 0.61) and WTA (error rate = 0.45). The final comparative analysis of classifiers based on the total error rate across all atmospheric corrections and pansharpening methods yielded the following reliability order: MXL > WTA > MHD > ACE > MD > CEM = MF > SAM > MTMF = TCIMF > OSP > MTTCIMF. The findings of the current study suggested that for VHR visible near-infrared (VNIR) mapping of facies, FLAASH was the best atmospheric correction, while MXL may deliver reliable thematic classification. Moreover, an extensive account of the varying exertions of each processing scheme is discussed, and could be transferable when compared against other VHR VNIR mapping methods. [ABSTRACT FROM AUTHOR]

Featured Application: Satellite image processing for change detection, target recognition, classification, map application, visual image analysis, etc. Pansharpening (PS) is a process used to generate high-resolution multispectral (MS) images from high-spatial-resolution panchromatic (PAN) and high-spectral-resolution multispectral images. In this paper, we propose a method for pansharpening by focusing on a compressed sensing (CS) technique. The spectral reproducibility of the CS technique is high due to its image reproducibility, but the reproduced image is blurry. Although methods of complementing this incomplete reproduction have been proposed, it is known that the existing method may cause ringing artifacts. On the other hand, component substitution is another technique used for pansharpening. It is expected that the spatial resolution of the images generated by this technique will be as high as that of the high-resolution PAN image, because the technique uses the corrected intensity calculated from the PAN image. Based on these facts, the proposed method fuses the intensity obtained by the component substitution method and the intensity obtained by the CS technique to move the spatial resolution of the reproduced image close to that of the PAN image while reducing the spectral distortion. Experimental results showed that the proposed method can reduce spectral distortion and maintain spatial resolution better than the existing methods. [ABSTRACT FROM AUTHOR]

Pansharpening aims at fusing a low-resolution multiband optical (MBO) image, such as a multispectral or a hyperspectral image, with the associated high-resolution panchromatic (PAN) image to yield a high spatial resolution MBO image. Though having achieved superior performances to traditional methods, existing convolutional neural network (CNN)-based pansharpening approaches are still faced with two challenges: alleviating the phenomenon of spectral distortion and improving the interpretation abilities of pansharpening CNNs. In this work, we develop a novel spectral-aware pansharpening neural network (SA-PNN). On the one hand, SA-PNN employs a network structure composed of a detail branch and an approximation branch, which is consistent with the detail injection framework; on the other hand, SA-PNN strengthens processing along the spectral dimension by using a spectral-aware strategy, which involves spatial feature transforms (SFTs) coupling the approximation branch with the detail branch as well as 3D convolution operations in the approximation branch. Our method is evaluated with experiments on real-world multispectral and hyperspectral datasets, verifying its excellent pansharpening performance. [ABSTRACT FROM AUTHOR]

Pansharpening is a method applied for the generation of high-spatial-resolution multispectral (MS) images using panchromatic (PAN) and multi-spectral images. A common challenge in pansharpening is to reduce the spectral distortion caused by increasing the resolution. In this paper, we propose a method for reducing the spectral distortion based on the intensity-hue-saturation (IHS) method targeting satellite images. The IHS method improves the resolution of an RGB image by replacing the intensity of the low-resolution RGB image with that of the high-resolution PAN image. The spectral characteristics of the PAN and MS images are different, and this difference may cause spectral distortion in the pansharpened image. Although many solutions for reducing spectral distortion using a modeled spectrum have been proposed, the quality of the outcomes obtained by these approaches depends on the image dataset. In the proposed technique, we model a low-spatial-resolution PAN image according to a relative spectral response graph, and then the corrected intensity is calculated using the model and the observed dataset. Experiments were conducted on three IKONOS datasets, and the results were evaluated using some major quality metrics. This quantitative evaluation demonstrated the stability of the pansharpened images and the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Featured Application: High resolution pansharpened images are used for detailed land use and land cover mapping. Preservation of spectral and spatial information is an important requirement for most quantitative remote sensing applications. In this study, we use image quality metrics to evaluate the performance of several image fusion techniques to assess the spectral and spatial quality of pansharpened images. We evaluated twelve pansharpening algorithms in this study; the Local Mean and Variance Matching (IMVM) algorithm was the best in terms of spectral consistency and synthesis followed by the ratio component substitution (RCS) algorithm. Whereas the IMVM and RCS image fusion techniques showed better results compared to other pansharpening methods, it is pertinent to highlight that our study also showed the credibility of other pansharpening algorithms in terms of spatial and spectral consistency as shown by the high correlation coefficients achieved in all methods. We noted that the algorithms that ranked higher in terms of spectral consistency and synthesis were outperformed by other competing algorithms in terms of spatial consistency. The study, therefore, concludes that the selection of image fusion techniques is driven by the requirements of remote sensing application and a careful trade-off is necessary to account for the impact of scene radiometry, image sharpness, spatial and spectral consistency, and computational overhead. [ABSTRACT FROM AUTHOR]

The RGBW color filter arrays (CFA), also known as CFA2.0, contains R, G, B, and white (W) pixels. It is a 4 × 4 pattern that has 8 white pixels, 4 green pixels, 2 red pixels, and 2 blue pixels. The pattern repeats itself over the whole image. In an earlier conference paper, we cast the demosaicing process for CFA2.0 as a pansharpening problem. That formulation is modular and allows us to insert different pansharpening algorithms for demosaicing. New algorithms in interpolation and demosaicing can also be used. In this paper, we propose a new enhancement of our earlier approach by integrating a deep learning-based algorithm into the framework. Extensive experiments using IMAX and Kodak images clearly demonstrated that the new approach improved the demosaicing performance even further. [ABSTRACT FROM AUTHOR]

Although Worldview-2 (WV) images (non-pansharpened) have 2-m resolution, the re-visit times for the same areas may be seven days or more. In contrast, Planet images are collected using small satellites that can cover the whole Earth almost daily. However, the resolution of Planet images is 3.125 m. It would be ideal to fuse these two satellites images to generate high spatial resolution (2 m) and high temporal resolution (1 or 2 days) images for applications such as damage assessment, border monitoring, etc. that require quick decisions. In this paper, we evaluate three approaches to fusingWorldview (WV) and Planet images. These approaches are known as Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM), Flexible Spatiotemporal Data Fusion (FSDAF), and Hybrid Color Mapping (HCM), which have been applied to the fusion of MODIS and Landsat images in recent years. Experimental results using actual Planet andWorldview images demonstrated that the three aforementioned approaches have comparable performance and can all generate high quality prediction images. [ABSTRACT FROM AUTHOR]

Numerous pansharpening methods were proposed in recent decades for fusing low-spatial-resolution multispectral (MS) images with high-spatial-resolution (HSR) panchromatic (PAN) bands to produce fused HSR MS images, which are widely used in various remote sensing tasks. The effect of misregistration between MS and PAN bands on quality of fused products has gained much attention in recent years. An improved method for misaligned MS and PAN imagery is proposed, through two improvements made on a previously published method named RMI (reduce misalignment impact). The performance of the proposed method was assessed by comparing with some outstanding fusion methods, such as adaptive Gram-Schmidt and generalized Laplacian pyramid. Experimental results show that the improved version can reduce spectral distortions of fused dark pixels and sharpen boundaries between different image objects, as well as obtain similar quality indexes with the original RMI method. In addition, the proposed method was evaluated with respect to its sensitivity to misalignments between MS and PAN bands. It is certified that the proposed method is more robust to misalignments between MS and PAN bands than the other methods. [ABSTRACT FROM AUTHOR]

Sensitivity to weather conditions, and specially to clouds, is a severe limiting factor to the use of optical remote sensing for Earth monitoring applications. A possible alternative is to benefit from weather-insensitive synthetic aperture radar (SAR) images. In many real-world applications, critical decisions are made based on some informative optical or radar features related to items such as water, vegetation or soil. Under cloudy conditions, however, optical-based features are not available, and they are commonly reconstructed through linear interpolation between data available at temporally-close time instants. In this work, we propose to estimate missing optical features through data fusion and deep-learning. Several sources of information are taken into account--optical sequences, SAR sequences, digital elevation model--so as to exploit both temporal and cross-sensor dependencies. Based on these data and a tiny cloud-free fraction of the target image, a compact convolutional neural network (CNN) is trained to perform the desired estimation. To validate the proposed approach, we focus on the estimation of the normalized difference vegetation index (NDVI), using coupled Sentinel-1 and Sentinel-2 time-series acquired over an agricultural region of Burkina Faso from May-November 2016. Several fusion schemes are considered, causal and non-causal, single-sensor or joint-sensor, corresponding to different operating conditions. Experimental results are very promising, showing a significant gain over baseline methods according to all performance indicators. [ABSTRACT FROM AUTHOR]

Coastline extraction is a fundamental work for coastal resource management, coastal environmental protection and coastal sustainable development. Due to the free access and long-term record, Landsat series images have the potential to be used for coastline extraction. However, dynamic features of different types of coastlines (e.g., rocky, sandy, artificial), caused by sea level fluctuation from tidal, storm and reclamation, make it difficult to be accurately extracted with coarse spatial resolution, e.g., 30 m, of Landsat images. To access this problem, we analyze the performance of coastline extraction by integrating downscaling, pansharpening and water index approaches in increasing the accuracy of coastline extraction from the latest Landsat-8 Operational Land Imager (OLI) imagery. In order to prove the availability of the proposed method, we designed three strategies: (1) Strategy 1 uses the traditional water index method to extract coastline directly from original 30 m Landsat-8 OLI multispectral (MS) image; (2) Strategy 2 extracts coastlines from 15 m fused MS images generated by integrating 15 m panchromatic (PAN) band and 30 m MS image with ten pansharpening algorithms; (3) Strategy 3 first downscales the PAN band to a finer spatial resolution (e.g., 7.5 m) band, and then extracts coastlines from pansharpened MS images generated by integrating downscaled spatial resolution PAN band and 30 m MS image with ten pansharpening algorithms. Using the coastline extracted from ZiYuan-3 (ZY-3) 5.8 m MS image as reference, accuracies of coastlines extracted from MS images in three strategies were validated visually and quantitatively. The results show that, compared with coastline extracted directly from 30 m Landsat-8 MS image (strategy 1), strategy 3 achieves the best accuracies with optimal mean net shoreline movement (MNSM) of -2.54 m and optimal mean absolute difference (MAD) of 11.26 m, followed by coastlines extracted in strategy 2 with optimal MNSM of -4.23 m and optimal MAD of 13.54 m. Further comparisons with single-band thresholding (Band 6), AWEI, and ISODATA also confirmed the superiority of strategy 3. For the various used pansharpening algorithms, five multiresolution analysis MRA-based pansharpening algorithms are more efficient than the component substitution CS-based pansharpening algorithms for coastline extraction from Landsat-8 OLI imagery. Among the five MRA-based fusion methods, the coastlines extracted from the fused images generated by Indusion, additive à trous wavelet transform (ATWT) and additive wavelet luminance proportional (AWLP) produced the most accurate and visually realistic representation. Therefore, pansharpening approaches can improve the accuracy of coastline extraction from Landsat-8 OLI imagery, and downscaling the PAN band to finer spatial resolution is able to further improve the coastline extraction accuracy, especially in crenulated coasts. [ABSTRACT FROM AUTHOR]

The standard ratio of spatial resolution between bands for high resolution satellites is 1:4, which is typical when combining images obtained from the same sensor. However, the cost of simultaneously purchasing a set of panchromatic and multispectral images is still relatively high. There is therefore a need to develop methods of data fusion of very high resolution panchromatic imagery with low-cost multispectral data (e.g., Landsat). Combining high resolution images with low resolution images broadens the scope of use of satellite data, however, it is also accompanied by the problem of a large ratio between spatial resolutions, which results in large spectral distortions in the merged images. The authors propose a modification of the panchromatic image in such a way that it includes the spectral and spatial information from both the panchromatic and multispectral images to improve the quality of spectral data integration. This fusion is done based on a weighted average. The weight is determined using a coefficient, which determines the ratio of the amount of information contained in the corresponding pixels of the integrated images. The effectiveness of the author's algorithm had been tested for six of the most popular fusion methods. The proposed methodology is ideal mainly for statistical and numerical methods, especially Principal Component Analysis and Gram-Schmidt. The author's algorithm makes it possible to lower the root mean square error by up to 20% for the Principal Component Analysis. The spectral quality was also increased, especially for the spectral bands extending beyond the panchromatic image, where the correlation rose by 18% for the Gram-Schmidt orthogonalization. [ABSTRACT FROM AUTHOR]

The component substitution (CS) pansharpening methods have been developed for almost three decades and have become better understood recently by generalizing them into one framework. However, few studies focus on the statistical assumptions implicit in the CS methods. This paper reveals their implicit statistical assumptions from a Bayesian data fusion framework and suggests best practices for histogram matching of the panchromatic image to the intensity image, a weighted summation of the multispectral images, to better satisfy these assumptions. The purpose of histogram matching was found to make the difference between the high-resolution panchromatic and intensity images as small as possible, as one implicit assumption claims their negligible difference. The statistical relationship between the high-resolution panchromatic and intensity images and the relationship between their corresponding low-resolution images are the same, as long as the low resolution panchromatic image is derived by considering the modulation transfer functions of the multispectral sensors. Hence, the histogram-matching equation should be derived from the low-resolution panchromatic and intensity images, but not derived from the high-resolution panchromatic and expanded low-resolution intensity images. Experiments using three example CS methods, each using the two different histogram-matching equations, was conducted on the four-band QuickBird and eight-band WorldView-2 top-of-atmosphere reflectance data. The results verified the best practices and showed that the histogram-matching equation derived from the high-resolution panchromatic and expanded low-resolution intensity images provides more-blurred histogram-matched panchromatic image and, hence less-sharpened pansharpened images than that derived from the low-resolution image pair. The usefulness of the assumptions revealed in this study for method developers is discussed. For example, the CS methods can be improved by satisfying the assumptions better, e.g., classifying the images into homogenous areas before pansharpening, and by changing the assumptions to be more general to address their deficiencies. [ABSTRACT FROM AUTHOR]

Ecosystems provide a wide variety of useful resources that enhance human welfare, but these resources are declining due to climate change and anthropogenic pressure. In this work, three vulnerable ecosystems, including shrublands, coastal areas with dunes systems and areas of shallow water, are studied. As far as these resources' reduction is concerned, remote sensing and image processing techniques could contribute to the management of these natural resources in a practical and cost-effective way, although some improvements are needed for obtaining a higher quality of the information available. An important quality improvement is the fusion at the pixel level. Hence, the objective of this work is to assess which pansharpening technique provides the best fused image for the different types of ecosystems. After a preliminary evaluation of twelve classic and novel fusion algorithms, a total of four pansharpening algorithms was analyzed using six quality indices. The quality assessment was implemented not only for the whole set of multispectral bands, but also for the subset of spectral bands covered by the wavelength range of the panchromatic image and outside of it. A better quality result is observed in the fused image using only the bands covered by the panchromatic band range. It is important to highlight the use of these techniques not only in land and urban areas, but a novel analysis in areas of shallow water ecosystems. Although the algorithms do not show a high difference in land and coastal areas, coastal ecosystems require simpler algorithms, such as fast intensity hue saturation, whereas more heterogeneous ecosystems need advanced algorithms, as weighted wavelet 'à trous' through fractal dimension maps for shrublands and mixed ecosystems. Moreover, quality map analysis was carried out in order to study the fusion result in each band at the local level. Finally, to demonstrate the performance of these pansharpening techniques, advanced Object-Based (OBIA) support vector machine classification was applied, and a thematic map for the shrubland ecosystem was obtained, which corroborates wavelet 'à trous' through fractal dimension maps as the best fusion algorithm for this ecosystem. [ABSTRACT FROM AUTHOR]

Since WorldView-2 (WV-2) images are widely used in various fields, there is a high demand for the use of high-quality pansharpened WV-2 images for different application purposes. With respect to the novelty of the WV-2 multispectral (MS) and panchromatic (PAN) bands, the performances of eight state-of-art pan-sharpening methods for WV-2 imagery including six datasets from three WV-2 scenes were assessed in this study using both quality indices and information indices, along with visual inspection. The normalized difference vegetation index, normalized difference water index, and morphological building index, which are widely used in applications related to land cover classification, the extraction of vegetation areas, buildings, and water bodies, were employed in this work to evaluate the performance of different pansharpening methods in terms of information presentation ability. The experimental results show that the Haze-and Ratio-based, adaptive Gram-Schmidt, Generalized Laplacian pyramids (GLP) methods using enhanced spectral distortion minimal model and enhanced context-based decision model methods are good choices for producing fused WV-2 images used for image interpretation and the extraction of urban buildings. The two GLP-based methods are better choices than the other methods, if the fused images will be used for applications related to vegetation and water-bodies. [ABSTRACT FROM AUTHOR]

State-of-the-art pansharpening methods generally inject the spatial structures of a high spatial resolution (HR) panchromatic (PAN) image into the corresponding low spatial resolution (LR) multispectral (MS) image by an injection model. In this paper, a novel pansharpening method with an edge-preserving guided filter based on three-layer decomposition is proposed. In the proposed method, the PAN image is decomposed into three layers: A strong edge layer, a detail layer, and a low-frequency layer. The edge layer and detail layer are then injected into the MS image by a proportional injection model. In addition, two new quantitative evaluation indices, including the modified correlation coefficient (MCC) and the modified universal image quality index (MUIQI) are developed. The proposed method was tested and verified by IKONOS, QuickBird, and Gaofen (GF)-1 satellite images, and it was compared with several of state-of-the-art pansharpening methods from both qualitative and quantitative aspects. The experimental results confirm the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Pansharpening algorithms fuse higher spatial resolution panchromatic with lower spatial resolution multispectral imagery to create higher spatial resolution multispectral images. The free-availability and systematic global acquisition of Landsat 8 data indicate an expected need for global coverage and so computationally efficient Landsat 8 pansharpening. This study adapts and evaluates the established, and relatively computationally inexpensive, Brovey and context adaptive Gram Schmidt component substitution (CS) pansharpening methods for application to the Landsat 8 15 m panchromatic and 30 m red, green, blue, and near-infrared bands. The intensity images used by these CS pansharpening methods are derived as a weighted linear combination of the multispectral bands in three different ways using band spectral weights set (i) equally as the reciprocal of the number of bands; (ii) using fixed Landsat 8 spectral response function based (SRFB) weights derived considering laboratory spectra; and (iii) using image specific spectral weights derived by regression between the multispectral and the degraded panchromatic bands. The spatial and spectral distortion and computational cost of the different methods are assessed using Landsat 8 test images acquired over agricultural scenes in South Dakota, China, and India. The results of this study indicate that, for global Landsat 8 application, the context adaptive Gram Schmidt pansharpening with an intensity image defined using the SRFB spectral weights is appropriate. The context adaptive Gram Schmidt pansharpened results had lower distortion than the Brovey results and the least distortion was found using intensity images derived using the SRFB and image specific spectral weights but the computational cost using the image specific weights was greater than the using the SRFB weights. Recommendations for large area Landsat 8 pansharpening application are described briefly and the SRFB spectral weights are provided so users may implement computationally inexpensive Landsat 8 pansharpening themselves. [ABSTRACT FROM AUTHOR]

Pansharpening is an important technique that produces high spatial resolution multispectral (MS) images by fusing low spatial resolution MS images and high spatial resolution panchromatic (PAN) images of the same area. Although numerous successful image fusion algorithms have been proposed in the last few decades to reduce the spectral distortions in fused images, few of these take into account the spectral distortions caused by mixed MS sub-pixels (MSPs). Typically, the fused versions of MSPs remain mixed, although some of the MSPs correspond to pure PAN pixels. Due to the significant spectral differences between vegetation and non-vegetation (VNV) objects, the fused versions of MSPs near VNV boundaries cause blurred VNV boundaries and significant spectral distortions in the fused images. In order to reduce the spectral distortions, an improved version of the haze- and ratio-based fusion method is proposed to realize the spectral un-mixing of MSPs near VNV boundaries. In this method, the MSPs near VNV boundaries are identified first. The identified MSPs are then defined as either pure vegetation or non-vegetation pixels according to the categories of the corresponding PAN pixels. Experiments on WorldView-2 and IKONOS images of urban areas using the proposed method yielded fused images with significantly clearer VNV boundaries and smaller spectral distortions than several other currently-used image fusion methods. [ABSTRACT FROM AUTHOR]

Component substitution (CS) and multi-resolution analysis (MRA) are the two basic categories in the extended general image fusion (EGIF) framework for fusing panchromatic (Pan) and multispectral (MS) images. Despite of the method diversity, there are some unaddressed questions and contradictory conclusions about fusion. For example, is the spatial enhancement of CS methods better than MRA methods? Is spatial enhancement and spectral preservation competitive? How to achieve spectral consistency defined by Wald et al. in 1997? In their definition any synthetic image should be as identical as possible to the original image once degraded to its original resolution. To answer these questions, this research first finds out that all the CS and MRA methods can be derived from the Bayesian fusion method by adjusting a weight parameter to balance contributions from the spatial injection and spectral preservation models. The spectral preservation model assumes a Gaussian distribution of the desired high-resolution MS images, with the up-sampled low-resolution MS images comprising the mean value. The spatial injection model assumes a linear correlation between Pan and MS images. Thus the spatial enhancement depends on the weight parameter but is irrelevant of which category (i.e., MRA or CS) the method belongs to. This paper then adds a spectral consistency model in the Bayesian fusion framework to guarantee Wald's spectral consistency with regard to arbitrary sensor point spread function. Although the spectral preservation in the EGIF methods is competitive to spatial enhancement, the Wald's spectral consistency property is complementary with spatial enhancement. We conducted experiments on satellite images acquired by the QuickBird and WorldView-2 satellites to confirm our analysis, and found that the performance of the traditional EGIF methods improved significantly after adding the spectral consistency model. [ABSTRACT FROM AUTHOR]

This study evaluated the effects of image pansharpening on Vegetation Indices (VIs), and found that pansharpening was able to downscale single-date and multi-temporal Landsat 8 VI data without introducing significant distortions in VI values. Four fast pansharpening methods--Fast Intensity-Hue-Saturation (FIHS), Brovey Transform (BT), Additive Wavelet Transform (AWT), and Smoothing Filter-based Intensity Modulation (SFIM)--and two VIs--Normalized Difference Vegetation Index (NDVI) and Simple Ratio (SR)--were tested. The NDVI and SR formulas were both found to cause some spatial information loss in the pansharpened multispectral (MS) bands, and this spatial information loss from VI transformations was not specific to Landsat 8 imagery (it will occur for any type of imagery). BT, SFIM, and other similar pansharpening methods that inject spatial information from the panchromatic (Pan) band by multiplication, lose all of the injected spatial information after the VI calculations. FIHS, AWT, and other similar pansharpening methods that inject spatial information by addition, lose some spatial information from the Pan band after VI calculations as well. Nevertheless, for all of the single- and multi-date VI images, the FIHS and AWT pansharpened images were more similar to the higher resolution reference data than the unsharpened VI images were, indicating that pansharpening was effective in downscaling the VI data. FIHS best enhanced the spectral and spatial information of the single-date and multi-date VI images, followed by AWT, and neither significantly over- or under-estimated VI values. [ABSTRACT FROM AUTHOR]

Compared with hardware upgrading, pansharpening is a low-cost way to acquire high-quality images, which usually combines multispectral images (MS) in low spatial resolution with panchromatic images (PAN) in high spatial resolution. This paper proposes a pixel-dependent spatial-detail injection network (PDSDNet). Based on a dynamic filter network, PDSDNet constructs nonlinear mapping of the simulated panchromatic band from low-resolution multispectral bands through filtering convolution regression. PDSDNet reduces the possibility of spectral distortion and enriches spatial details by improving the similarity between the simulated panchromatic band and the real panchromatic band. Moreover, PDSDNet assumes that if an ideal multispectral image that has the same resolution with the panchromatic image exists, each band of it should have the same spatial details as in the panchromatic image. Thus, the details we fill into each multispectral band are the same and they can be extracted effectively in one pass. Experimental results demonstrate that PDSDNet can generate high-quality fusion images with multispectral images and panchromatic images. Compared with BDSD, MTF-GLP-HPM-PP, and PanNet, which are widely applied on IKONOS, QuickBird, and WorldView-3 datasets, pansharpened images of the proposed method have rich spatial details and present superior visual effects without noticeable spectral and spatial distortion. [ABSTRACT FROM AUTHOR]

The pansharpening (PS) of remote-sensing images aims to fuse a high-resolution panchromatic image with several low-resolution multispectral images for obtaining a high-resolution multispectral image. In this work, a two-stage PS model is proposed by integrating the ideas of component replacement and the variational method. The global sparse gradient of the panchromatic image is extracted by variational method, and the weight function is constructed by combining the gradient of multispectral image in which the global sparse gradient can provide more robust gradient information. Furthermore, we refine the results in order to reduce spatial and spectral distortions. Experimental results show that our method had high generalization ability for QuickBird, Gaofen-1, and WorldView-4 satellite data. Experimental results evaluated by seven metrics demonstrate that the proposed two-stage method enhanced spatial details subjective visual effects better than other state-of-the-art methods do. At the same time, in the process of quantitative evaluation, the method in this paper had high improvement compared with that other methods, and some of them can reach a maximal improvement of 60%. [ABSTRACT FROM AUTHOR]

Multispectral (MS) pansharpening is crucial to improve the spatial resolution of MS images. MS pansharpening has the potential to provide images with high spatial and spectral resolutions. Pansharpening technique based on deep learning is a topical issue to deal with the distortion of spatio-spectral information. To improve the preservation of spatio-spectral information, we propose a novel three-stage detail injection pansharpening network (TDPNet) for remote sensing images. First, we put forward a dual-branch multiscale feature extraction block, which extracts four scale details of panchromatic (PAN) images and the difference between duplicated PAN and MS images. Next, cascade cross-scale fusion (CCSF) employs fine-scale fusion information as prior knowledge for the coarse-scale fusion to compensate for the lost information during downsampling and retain high-frequency details. CCSF combines the fine-scale and coarse-scale fusion based on residual learning and prior information of four scales. Last, we design a multiscale detail compensation mechanism and a multiscale skip connection block to reconstruct injecting details, which strengthen spatial details and reduce parameters. Abundant experiments implemented on three satellite data sets at degraded and full resolutions confirm that TDPNet trades off the spectral information and spatial details and improves the fidelity of sharper MS images. Both the quantitative and subjective evaluation results indicate that TDPNet outperforms the compared state-of-the-art approaches in generating MS images with high spatial resolution. [ABSTRACT FROM AUTHOR]

Intensity-Hue-Saturation (IHS), Brovey Transform (BT), and Smoothing-Filter-Based-Intensity Modulation (SFIM) algorithms were used to pansharpen GeoEye-1 imagery. The pansharpened images were then segmented in Berkeley Image Seg using a wide range of segmentation parameters, and the spatial and spectral accuracy of image segments was measured. We found that pansharpening algorithms that preserve more of the spatial information of the higher resolution panchromatic image band (i.e., IHS and BT) led to more spatially-accurate segmentations, while pansharpening algorithms that minimize the distortion of spectral information of the lower resolution multispectral image bands (i.e., SFIM) led to more spectrally-accurate image segments. Based on these findings, we developed a new IHS-SFIM combination approach, specifically for object-based image analysis (OBIA), which combined the better spatial information of IHS and the more accurate spectral information of SFIM to produce image segments with very high spatial and spectral accuracy. [ABSTRACT FROM AUTHOR]