Search

Benefitting from the strong feature extraction capability of deep learning, infrared and visible image fusion has made a great progress. Since infrared and visible images are obtained by different sensors with different imaging mechanisms, there exists domain discrepancy, which becomes stumbling block for effective fusion. In this paper, we propose a novel self-supervised feature adaption framework for infrared and visible image fusion. We implement a self-supervised strategy that facilitates the backbone network to extract features with adaption while retaining the vital information by reconstructing the source images. Specifically, we preliminary adopt an encoder network to extract features with adaption. Then, two decoders with attention mechanism blocks are utilized to reconstruct the source images in a self-supervised way, forcing the adapted features to contain vital information of the source images. Further, considering the case that source images contain low-quality information, we design a novel infrared and visible image fusion and enhancement model, improving the fusion method's robustness. Experiments are constructed to evaluate the proposed method qualitatively and quantitatively, which show that the proposed method achieves the state-of-art performance comparing with existing infrared and visible image fusion methods. Results are available at https://github.com/zhoafan/SFA-Fuse. • Domain discrepancy between infrared and visible images reduces fusion performance. • Self-supervised mechanism is designed for infrared and visible image fusion. • The novel fusion and enhancement model is more suitable for practical application. • Our method performs better than other representative fusion methods. [ABSTRACT FROM AUTHOR]

Common non-focused areas are often present in multi-focus images due to the limitation of the number of focused images. This factor severely degrades the fusion quality of multi-focus images. To address this problem, we propose a novel end-to-end multi-focus image fusion with a natural enhancement method based on deep convolutional neural network (CNN). Several end-to-end CNN architectures that are specifically adapted to this task are first designed and researched. On the basis of the observation that low-level feature extraction can capture low-frequency content, whereas high-level feature extraction effectively captures high-frequency details, we further combine multi-level outputs such that the most visually distinctive features can be extracted, fused, and enhanced. In addition, the multi-level outputs are simultaneously supervised during training to boost the performance of image fusion and enhancement. Extensive experiments show that the proposed method can deliver superior fusion and enhancement performance than the state-of-the-art methods in the presence of multi-focus images with common non-focused areas, anisotropic blur, and misregistration. [ABSTRACT FROM AUTHOR]

Highlights • An efficient multi-focus fusion method based on local binary patterns is proposed. • The frequent loss of low-contrast area information is successfully avoided. • The proposed algorithm outperforms state-of-the-art image fusion algorithms. Abstract Multi-focus image fusion is to integrate the partially focused images into one single image which is focused everywhere. Nowadays, it has become an important research topic due to the applications in more and more scientific fields. However, preserving more information of the low-contrast area in the focus area and maintaining the edge information are two challenges for existing approaches. In this paper, we address these two challenges with presenting a simple yet efficient multi-focus fusion method based on local binary pattern (LBP). In our algorithm, we measure the clarity using the LBP metric and construct the initial weight map. And then we use the connected area judgment strategy (CAJS) to reduce the noise in the initial map. Afterwards, the two source images are fused together by weighted arranging. The experimental results validate that the proposed algorithm outperforms state-of-the-art image fusion algorithms in both qualitative and quantitative evaluations, especially when dealing with low contrast regions and edge information. [ABSTRACT FROM AUTHOR]

Most existing image fusion methods assume that at least one input image contains high-quality information at any place of an observed scene. Thus, these fusion methods will fail if every input image is degraded. To address this issue, this study proposes a novel fusion framework that integrates image fusion based on spectral total variation (TV) method and image enhancement. For spatially varying multiscale decompositions generated by the spectral TV framework, this study verifies that the decomposition components can be modeled efficiently by tailed $\alpha$-stable-based random variable distribution (TRD) rather than the commonly used Gaussian distribution. Consequently, salience and match measures based on TRD are proposed to fuse each sub-band decomposition. The spatial intensity information is also adopted to fuse the remainder of the image decomposition components. A sub-band adaptive gain function family based on TV spectrum and space variation is constructed for fused multiscale decompositions to enhance fused image simultaneously. Finally, numerous experiments with various multisensor image pairs are conducted to evaluate the proposed method. Experimental results show that even if the input images are degraded, the fused image obtained by the proposed method achieves significant improvement in terms of edge details and contrast while extracting the main features of the input images, thereby achieving better performance compared with the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Medical image fusion aims at integrating information from multimodality medical images to obtain a more complete and accurate description of the same object, which provides an easy access for image-guided medical diagnostic and treatment. Unfortunately, medical images are often corrupted by noise in acquisition or transmission, and the noise signal is easily mistaken for a useful characterization of the image, making the fusion effect drop significantly. Thus, the existence of noise presents a great challenge for most of traditional image fusion methods. To address this problem, an effective variation model for multimodality medical image fusion and denoising is proposed. First, a multiscale alternating sequential filter is exploited to extract the useful characterizations (e.g., details and edges) from noisy input medical images. Then, a recursive filtering-based weight map is constructed to guide the fusion of main features of input images. Additionally, total variation (TV) constraint is developed by constructing an adaptive fractional order $p$ based on the local contrast of fused image, further effectively suppressing noise while avoiding the staircase effect of the TV. The experimental results indicate that the proposed method performs well with both noisy and normal medical images, outperforming conventional methods in terms of fusion quality and noise reduction. [ABSTRACT FROM PUBLISHER]

Noise is easily mistaken as useful features of input images, and therefore, significantly reducing image fusion quality. In this paper, we propose a novel gradient entropy metric and p -Laplace diffusion constraint-based method. Specifically, the method is based on the matrix of structure tensor to fuse the gradient information. To minimize the negative effects of noise on the selections of image features, the gradient entropy metric is proposed to construct the weight for each gradient of input images. Particularly, the local adaptive p -Laplace diffusion constraint is constructed to further suppress noise when rebuilding the fused image from the fused gradient field. Experimental results show that the proposed method effectively preserves edge detail features of multispectral images while suppressing noise, achieving an optimal visual effect and more comprehensive quantitative assessments compared to other existing methods. [ABSTRACT FROM AUTHOR]