Search

Medical multi-modal retrieval aims to provide doctors with similar medical images from different modalities, which can greatly promote the efficiency and accuracy of clinical diagnosis. However, most existing medical retrieval methods hardly support the retrieval of multi-modal medical images, i.e., the number of modalities is greater than 2, and just convert retrieval to classification or clustering. It futilely breaks the gap between the visual information and the semantic information in different medical image modalities. To solve the problem, a Supervised Contrast Learning method based on a Multiple Pseudo-Siamese network (SCL-MPS) is proposed for multi-modal medical image retrieval. In order to make the samples with semantic similarity close neighbors on Riemann manifold, the multiple constraints based on semantic consistency and modal invariance are designed in different forward stages of SCL-MPS. We theoretically demonstrate the feasibility of the designed constraints. Finally, experiments on four benchmark datasets (ADNI1, ADNI2, ADNI3, and OASIS3) show that SCL-MPS achieves state-of-the-art performance compared to 15 retrieval methods. Especially, SCL-MPS achieves a 100% mAP score in medical cross-modal retrieval on ADNI1. [ABSTRACT FROM AUTHOR]

Because of the limitations of convolution kernel, the traditional image segmentation network is not sufficient to obtain the context information, but the image segmentation task is very dependent on the context information. Transformer's linear input can just get enough context information. In this paper, we propose a transformer segmentation network hyperfusion transformer based on a pyramid structure. First, the model divides the single-scale coding form into several-different-scale coding forms, and then fuses the decoding results. Second, in order to ensure the specificity of the output characteristics of each branch, we orthogonalize the results of a variety of different scales. By orthogonalizing in pairs, we can ensure that the results obtained by different branches are not completely similar to a certain extent, and reduce the redundancy of branch information. On the two datasets, the method in this paper surpasses a variety of classical models under multiple evaluation indexes, confirming that it is an effective segmentation method. [ABSTRACT FROM AUTHOR]