Multispectral and hyperspectral image fusion via multi-model deep prior regularization.

Due to the difficulty of collecting a sufficient number of photons within a narrow spectral window, hyperspectral images often have low spatial resolution. In order to solve this problem without increasing hardware costs, a large number of image super-resolution algorithms have been proposed and have shown promising results. However, the performance of these algorithms is inconsistent in different datasets. To fix this situation, we proposed a multi-model deep prior regularization method for multispectral and hyperspectral image fusion. This method leverages mechanistic knowledge to combine various super-resolution results into a single algorithm, significantly improving the quality of the hyperspectral images. Specifically, we first proposed a variable residual mixed attention network. This network takes a low-resolution hyperspectral image and a high-resolution multispectral image as inputs, producing deep prior results from three different modes by adjusting spatial and channel attention. Then, we reconstructed the prior results along the spectral dimension and used them as regularization terms in our mechanistic model. We transformed the problem of finding the optimal solution of the model into solving the Sylvester equation and applied a fast solver for this equation to create the high-resolution hyperspectral images. Additionally, we used a genetic algorithm to determine the best hyperparameters for different deep prior results. Through validation on three datasets, we have demonstrated that our method performs better than existing methods in both quantitative and qualitative comparisons. [ABSTRACT FROM AUTHOR]