A hybrid model combining tensor and mutual information for multi-modal image registration

There are significant nonlinear intensity differences between multi-modal images. Moreover, the noise in these images will cause image degradation. Therefore, the automatic registration of multi-modal images is a challenging task. To address the two problems, this paper proposes a multi-modal image automatic registration method, which is divided into two stages: pre-registration and fine registration. In the pre-registration stage, an improved SIFT algorithm is used to roughly align multi-modal images. In the fine registration stage, the block Harris detector is first used to extract evenly distributed feature points on the pre-registered reference image. Then, the structure information in the multi-modal images is captured by the anisotropic structure tensor to construct a feature descriptor, which is robust to noise. Furthermore, a similarity criterion named TOMI (tensor orientation and mutual information) is proposed combining the tensor orientation parallelism and gradient mutual information. Finally, Multi-modal images (including Optical, LiDAR, SAR, and Map data) are used to evaluate the proposed algorithm. The experimental results show that the method proposed in this paper is robust to nonlinear intensity differences and noise, and the matching effect is superior.