Hybrid generative adversarial network based on a mixed attention fusion module for multi-modal MR image synthesis algorithm.

Recently, medical image synthesis has attracted the attention of an increasing number of researchers. However, most of current approaches suffer from the loss of multi-modal complementary information and thus fail to preserve the property of each modality, resulting in image distortion and texture detail loss. To alleviate this issue, a multi-modal magnetic resonance (MR) image synthesis algorithm based on a mixed attention fusion module in hybrid generative adversarial network is proposed. Firstly, a novel mixed attention fusion (MAF) module aggregating an adaptive fusion strategy (AFS) and a soft attention module is proposed to fuse the high-level semantic information and the low-level fine-grained feature at different scales between different layers to exploit rich representative complementary information adaptively. Subsequently, Resnet-bottlenect attention mechanism (Res-BAM) is designed to perform adaptive optimization and exploit mutual information while preserving the original property of each modality. Thereafter, the attention weight is inferred by a 1D channel feature map and a 2D spatial feature map, and multiplied with the original feature map in order to get the adaptive feature map, which is integrated with the original feature map in a residual connection to preserve the original property of each modality and prevent network degradation. Finally, the structural similarity (SSIM) and L 1 -norm are point-wise combined by an optimal weighting impact factor to preserve the high frequency information, brightness, color and SSIM, which are viewed as the original property of each modality. The experimental results demonstrate the superiority of our model on the state of the art in quantitative measures, reasonable visual quality and clinic significance. [ABSTRACT FROM AUTHOR]