Search

Due to the limitation of imaging equipment, it is difficult to acquire hyperspectral images (HSIs) with high spatial resolution directly. Existing approaches improve the resolution of HSIs by fusing multispectral image (MSI) and HSI. However, most of them are only feed-forward. They only learn low-resolution (LR) to high-resolution (HR) feature mappings without considering the ill-posedness of super-resolution tasks, leading to a large solution space of mapping functions and making it difficult to learn a complete mapping function. Moreover, there is a large resolution difference between HSI and MSI, and some up-sampling operations are inevitably employed in the network. Nevertheless, traditional upsampling methods only represent pixel points in a discrete way, failing to adequately restore the continuous spatial and spectral information. To this end, this article proposes a spatial-spectral implicit neural representation network for hyperspectral and multispectral image fusion (SS-INR). Inspired by the success of implicit neural representation (INR) in continuum reconstruction, we design spatial-INR and spectral-INR for spatial and spectral resolution reconstruction, respectively. SS-INR contains two processes: forward fusion (FF) and back-projection fusion (BPF). In the FF process, the input HSI is first spatially upsampled with spatial-INR to overcome spatial resolution differences while performing initial fusion with MSI. In the BPF process, we explore the spatial and spectral degradation processes and use them as prior knowledge for error correction. Extensive experiments on five public hyperspectral datasets demonstrate the effectiveness of SS-INR, and SS-INR achieves competitive results compared with existing state-of-the-art (SOTA) fusion methods. The source code for SS-INR will be released at https://github.com/wxy11-27/SS-INR.

Improving the spatial resolution of hyperspectral (HS) images is of great significance for the subsequent applications. As the multispectral (MS) image can provide abundant complementary land-cover spatial information, HS and MS image fusion (HMF) methods have become a mainstream to generate HS images with both high spatial and spectral resolution. HMF has witnessed rapid progress by leveraging dictionary learning (DicL) technique. However, existing approaches are highly sensitive to the image registration accuracy, and the reconstruction performance of the nonoverlapped spectral bands between HS and MS images is extremely limited. To alleviate the effect of image misregistration and enrich the spectral information of nonoverlapped bands, a general HMF DicL framework, which considers nonoverlapped spectral bands reconstruction and image misregistration, is proposed in this article. For registration error, the proposed method is rectified by the improved DicL, which can solve the problem of the spectral information matching gap existing in traditional HMF methods between HS image and MS image. Meanwhile, for nonoverlapped spectral bands reconstruction, a novel coefficient optimization strategy is adopted to improve the nonoverlapped bands reconstruction. Therefore, the registration error can be avoided to the greatest extent and the accuracy of nonoverlapped bands reconstruction can be effectively improved. Experiments both on simulated and real-world datasets demonstrate that the proposed method can effectively tackle the registration error problem and increase the HMF accuracy with different spectral ranges. Meanwhile, the proposed framework provides guidance significance for the DicL-based HMF methods with various constraints to improve the nonoverlapped bands reconstruction accuracy.