Integrating image and gene-data with a semi-supervised attention model for prediction of KRAS gene mutation status in non-small cell lung cancer.

KRAS is a pathogenic gene frequently implicated in non-small cell lung cancer (NSCLC). However, biopsy as a diagnostic method has practical limitations. Therefore, it is important to accurately determine the mutation status of the KRAS gene non-invasively by combining NSCLC CT images and genetic data for early diagnosis and subsequent targeted therapy of patients. This paper proposes a Semi-supervised Multimodal Multiscale Attention Model (S²MMAM). S²MMAM comprises a Supervised Multilevel Fusion Segmentation Network (SMF-SN) and a Semi-supervised Multimodal Fusion Classification Network (S²MF-CN). S²MMAM facilitates the execution of the classification task by transferring the useful information captured in SMF-SN to the S²MF-CN to improve the model prediction accuracy. In SMF-SN, we propose a Triple Attention-guided Feature Aggregation module for obtaining segmentation features that incorporate high-level semantic abstract features and low-level semantic detail features. Segmentation features provide pre-guidance and key information expansion for S²MF-CN. S²MF-CN shares the encoder and decoder parameters of SMF-SN, which enables S²MF-CN to obtain rich classification features. S²MF-CN uses the proposed Intra and Inter Mutual Guidance Attention Fusion (I²MGAF) module to first guide segmentation and classification feature fusion to extract hidden multi-scale contextual information. I²MGAF then guides the multidimensional fusion of genetic data and CT image data to compensate for the lack of information in single modality data. S²MMAM achieved 83.27% AUC and 81.67% accuracy in predicting KRAS gene mutation status in NSCLC. This method uses medical image CT and genetic data to effectively improve the accuracy of predicting KRAS gene mutation status in NSCLC. [ABSTRACT FROM AUTHOR]