Search

Manual segmentation of brain tumors using structural magnetic resonance imaging (MRI) is an arduous and time-consuming task. Therefore, automatic and robust segmentation will considerably influence neuro-oncological clinical trials by reducing excessive manual annotation time. Herein, we propose a deep learning model that automatically segments brain tumors even in cases of missing MRI sequences, which are common in practical clinical settings. To address this issue, we enhance a generative adversarial network (GAN) by incorporating a squeeze-and-excitation (SE) attention module into its generator and a PatchGAN into its discriminator. The SE module recalibrates channel responses by explicitly modeling interdependencies, enabling the network to focus on critical regions such as tumor areas. Our proposed generative model is optimized using a combination of adversarial, structural similarity, and mean absolute error losses to synthesize missing MRI sequences more effectively. This enhancement allows our model to synthesize the missing MRI sequence (fluid attenuated inversion recovery [FLAIR]) by leveraging information from other available sequences (T1-weighted, T2-weighted, or contrast-enhanced T1-weighted [T1ce]). For the segmentation task, we employ an optimized nnU-Net model, which is trained using existing sequences and evaluated using both available and synthesized sequences (including missing ones), mimicking real-world scenarios where often only limited MRI sequences are available or usable. Our findings reveal a notable enhancement in brain tumor segmentation, as indicated by a significant increase in overall the Dice similarity coefficient (DSC) from 0.688% (when FLAIR is missing) to 0.873% (when using synthesized FLAIR derived from T2). This improvement brings the segmentation performance closer to what was achieved when real FLAIR was available, where the DSC reaches 0.901%. Moreover, our synthesizing model was also tested on two additional datasets: the BraTS 2020 validation set and BraTS Africa 2023 training set, which produces results comparable to those of BraTS 2021, thereby proving its robustness and generalizability. In addition, the resulting tumor segmentations are subsequently employed to assess the response to treatment in cases where all sequences were available and when synthesis was employed, according to response assessment in neuro-oncology criteria. [Display omitted] [ABSTRACT FROM AUTHOR]

To develop a research overview of brain tumor classification using machine learning, we conducted a systematic review with a bibliometric analysis. Our systematic review and bibliometric analysis included 1747 studies of automated brain tumor detection using machine learning reported in the previous 5 years (2019–2023) from 679 different sources and authored by 6632 investigators. Bibliographic data were collected from the Scopus database, and a comprehensive bibliometric analysis was conducted using Biblioshiny and the R platform. The most productive and collaborative institutes, reports, journals, and countries were determined using citation analysis. In addition, various collaboration metrics were determined at the institute, country, and author level. Lotka's law was tested using the authors' performance. Analysis showed that the authors' publication trends followed Lotka's inverse square law. An annual publication analysis showed that 36.46% of the studies had been reported in 2022, with steady growth from previous years. Most of the cited authors had focused on multiclass classification and novel convolutional neural network models that are efficient for small training sets. A keyword analysis showed that "deep learning," "magnetic resonance imaging," "nuclear magnetic resonance imaging," and "glioma" appeared most often, proving that of the several brain tumor types, most studies had focused on glioma. India, China, and the United States were among the highest collaborative countries in terms of both authors and institutes. The University of Toronto and Harvard Medical School had the highest number of affiliations with 132 and 87 publications, respectively. [ABSTRACT FROM AUTHOR]

The performance of deep learning-based methods depends mainly on the availability of large-scale labeled learning data. However, obtaining precisely annotated examples is challenging in the medical domain. Although some semi-supervised deep learning methods have been proposed to train models with fewer labels, only a few studies have focused on the uncertainty caused by the low quality of the images and the lack of annotations. This paper addresses the above issues using Dempster-Shafer theory and deep learning: 1) a semi-supervised learning algorithm is proposed based on an image transformation strategy; 2) a probabilistic deep neural network and an evidential neural network are used in parallel to provide two sources of segmentation evidence; 3) Dempster's rule is used to combine the two pieces of evidence and reach a final segmentation result. Results from a series of experiments on the BraTS2019 brain tumor dataset show that our framework achieves promising results when only some training data are labeled. [ABSTRACT FROM AUTHOR]

Clinical diagnosis and treatment decisions rely upon the integration of patient-specific data with clinical reasoning. Cancer presents a unique context that influences treatment decisions, given its diverse forms of disease evolution. Biomedical imaging allows non-invasive assessment of diseases based on visual evaluations, leading to better clinical outcome prediction and therapeutic planning. Early methods of brain cancer characterization predominantly relied upon the statistical modeling of neuroimaging data. Driven by breakthroughs in computer vision, deep learning has become the de facto standard in medical imaging. Integrated statistical and deep learning methods have recently emerged as a new direction in the automation of medical practice unifying multi-disciplinary knowledge in medicine, statistics, and artificial intelligence. In this study, we critically review major statistical, deep learning, and probabilistic deep learning models and their applications in brain imaging research with a focus on MRI-based brain tumor segmentation. These results highlight that model-driven classical statistics and data-driven deep learning is a potent combination for developing automated systems in clinical oncology. [Display omitted] • Deep learning and statistical models used in brain tumor segmentation are reviewed. • Probabilistic deep learning and applications of hybrid methods are summarized. • The review is conducted from both a theory-driven and application perspective. • Challenges and future directions in neuroimaging analysis are discussed. [ABSTRACT FROM AUTHOR]

Brain tumor segmentation in multimodal MRI has great significance in clinical diagnosis and treatment. The utilization of multimodal information plays a crucial role in brain tumor segmentation. However, most existing methods focus on the extraction and selection of deep semantic features, while ignoring some features with specific meaning and importance to the segmentation problem. In this paper, we propose a brain tumor segmentation method based on the fusion of deep semantics and edge information in multimodal MRI, aiming to achieve a more sufficient utilization of multimodal information for accurate segmentation. The proposed method mainly consists of a semantic segmentation module, an edge detection module and a feature fusion module. In the semantic segmentation module, the Swin Transformer is adopted to extract semantic features and a shifted patch tokenization strategy is introduced for better training. The edge detection module is designed based on convolutional neural networks (CNNs) and an edge spatial attention block (ESAB) is presented for feature enhancement. The feature fusion module aims to fuse the extracted semantic and edge features, and we design a multi-feature inference block (MFIB) based on graph convolution to perform feature reasoning and information dissemination for effective feature fusion. The proposed method is validated on the popular BraTS benchmarks. The experimental results verify that the proposed method outperforms a number of state-of-the-art brain tumor segmentation methods. The source code of the proposed method is available at https://github.com/HXY-99/brats. • Proposes a brain tumor segmentation method by fusing semantic and edge features. • Presents a Swin Transformer-based semantic segmentation module with an SPD strategy. • Presents a CNN-based edge detection module and an edge spatial attention block. • Presents a graph convolution-based multi-feature inference block for feature fusion. • Achieves very promising results on the popular BraTS benchmarks. [ABSTRACT FROM AUTHOR]

• This paper introduces the mutual learning strategy train the brain tumor segmentation network, using the shallowest feature map to supervise the subsequent feature map of the network. using the deepest logits to supervise the previous shallow network's logits. The shallow feature map and deep logit supervise mutually and improve the accuracy of tumor sub-region segmentation. • This paper introduces the depth supervision to train this network, using the prediction of each up-sample layer is to deep supervise the training process for enlarging the receptive field to improve the overall segmentation accuracy. • A large number of experiments on BraTS dataset show that our method can effectively improve the accuracy of brain tumor segmentation and achieve the performance of SOTA. Deep learning methods have been successfully applied to Brain tumor segmentation. However, the extreme data imbalance exists in the different sub-regions of tumor, results in training the deep learning methods on these data will reduce the accuracy of segmentation. We introduce the deep mutual learning strategy to address the challenges, the proposed integrates transformer layers in both encoder and decoder of a U-Net architecture. In the network, using the prediction of up-sampled layer is to deep supervise the training process for enlarging the receptive field to extract features, the feature map of the shallowest layer supervises the subsequent feature map of layers to keep more edge information to guide the sub-region segmentation accuracy. the classification logits of the deepest layer supervise the previous layer of logits to get more semantic information for distinguish of tumor sub-regions. Furthermore, the feature map and the classification logits supervise mutually to improve the overall segmentation accuracy. The experimental results on benchmark dataset shows that our method has significant performance gain over existing methods. [ABSTRACT FROM AUTHOR]

Automated magnetic resonance imaging (MRI) segmentation plays an important role for the diagnosis and evaluation of brain tumors. However, it involves many challenges, including large variations in the shape and types of brain tumors, especially small ones that are difficult to diagnose early, and the complex border patterns that lead to missing data and low segmentation accuracy, putting specialists under time pressure. In this paper, we propose a new, lightweight network based on Unet by integrating skip connections with depthwise convolutional layers. This complementary technique allows us to improve the performance of the Unet model in explaining and analyzing medical images in a comprehensive and effective manner by transferring information from advanced layers to lower layers with greater quality. It helps avoid information loss and contributes to improving the network's ability to extract features better. Our suggested approach produced superior results compared to other Unet-like networks, as evidenced by the following metrics: accuracy of 99.82%, Intersection over union (IoU) of 83.34%, and Dice Similarity Coefficient (DSC) of 90.83%. Experimental findings demonstrated that the skip connections reconstructed in depthwise Convolution Bottleneck (DCB-Unet) integrated multi-scale contextual semantic information. In contrast to the majority of contemporary medical image segmentation models, our suggested approach more precisely recognizes organs, lesions, and segments. [ABSTRACT FROM AUTHOR]

Different brain tumor magnetic resonance imaging (MRI) modalities provide diverse tumor-specific information. Previous works have enhanced brain tumor segmentation performance by integrating multiple MRI modalities. However, multi-modal MRI data are often unavailable in clinical practice. An incomplete modality leads to missing tumor-specific information, which degrades the performance of existing models. Various strategies have been proposed to transfer knowledge from a full modality network (teacher) to an incomplete modality one (student) to address this issue. However, they neglect the fact that brain tumor segmentation is a structural prediction problem that requires voxel semantic relations. In this paper, we propose a Reconstruct Incomplete Relation Network (RIRN) that transfers voxel semantic relational knowledge from the teacher to the student. Specifically, we propose two types of voxel relations to incorporate structural knowledge: Class-relative relations (CRR) and Class-agnostic relations (CAR). The CRR groups voxels into different tumor regions and constructs a relation between them. The CAR builds a global relation between all voxel features, complementing the local inter-region relation. Moreover, we use adversarial learning to align the holistic structural prediction between the teacher and the student. Extensive experimentation on both the BraTS 2018 and BraTS 2020 datasets establishes that our method outperforms all state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• A novel knowledge distillation framework to compress brain tumor segmentation models. • Proposing the Dynamic Weighted Knowledge Distillation (DWKD) algorithm. • Introducing the Regularized Cross-Entropy (RCE) loss function to enhance the model's robustness. • We have confirmed that DWKD demonstrates superior capability in enhancing model interpretability compared to SKD. • Compared to SKD, our proposed method outperforms on BraTS 2019, BraTS 2020, and BraTS 2021 datasets. Automatic 3D MRI brain tumor segmentation holds a crucial position in the field of medical image analysis, contributing significantly to the clinical diagnosis and treatment of brain tumors. However, traditional 3D brain tumor segmentation methods often entail extensive parameters and computational demands, posing substantial challenges in model training and deployment. To overcome these challenges, this paper introduces a brain tumor segmentation framework based on knowledge distillation. This framework includes training a lightweight network by extracting knowledge from a well-established brain tumor segmentation network. Firstly, this framework replaces the conventional static knowledge distillation (SKD) with the proposed dynamic weighted knowledge distillation (DWKD). DWKD dynamically adjusts the distillation loss weights for each pixel based on the learning state of the student network. Secondly, to enhance the student network's generalization capability, this paper customizes a loss function for DWKD, known as regularized cross-entropy (RCE). RCE introduces controlled noise into the model, enhancing its robustness and diminishing the risk of overfitting. This controlled injection of noise aids in fortifying the model's robustness. Lastly, Empirical validation of the proposed methodology is conducted using two distinct backbone networks, namely Attention U-Net and Residual U-Net. Rigorous experimentation is executed across the BraTS 2019, BraTS 2020, and BraTS 2021 datasets. Experimental results demonstrate that DWKD exhibits significant advantages over SKD in enhancing the segmentation performance of the student network. Furthermore, when dealing with limited training data, the RCE method can further improve the student network's segmentation performance. Additionally, this paper quantitatively analyzes the number of concept detectors identified in network dissection. It assesses the impact of DWKD on model interpretability and finds that compared to SKD, DWKD can more effectively enhance model interpretability. The source code is available at https://github.com/YuBinLab-QUST/DWKD/. [ABSTRACT FROM AUTHOR]

The burgeoning field of brain health research increasingly leverages artificial intelligence (AI) to analyze and interpret neuroimaging data. Medical foundation models have shown promise of superior performance with better sample efficiency. This work introduces a novel approach towards creating 3-dimensional (3D) medical foundation models for multimodal neuroimage segmentation through self-supervised training. Our approach involves a novel two-stage pretraining approach using vision transformers. The first stage encodes anatomical structures in generally healthy brains from the large-scale unlabeled neuroimage dataset of multimodal brain magnetic resonance imaging (MRI) images from 41,400 participants. This stage of pertaining focuses on identifying key features such as shapes and sizes of different brain structures. The second pretraining stage identifies disease-specific attributes, such as geometric shapes of tumors and lesions and spatial placements within the brain. This dual-phase methodology significantly reduces the extensive data requirements usually necessary for AI model training in neuroimage segmentation with the flexibility to adapt to various imaging modalities. We rigorously evaluate our model, BrainSegFounder, using the Brain Tumor Segmentation (BraTS) challenge and Anatomical Tracings of Lesions After Stroke v2.0 (ATLAS v2.0) datasets. BrainSegFounder demonstrates a significant performance gain, surpassing the achievements of the previous winning solutions using fully supervised learning. Our findings underscore the impact of scaling up both the model complexity and the volume of unlabeled training data derived from generally healthy brains. Both of these factors enhance the accuracy and predictive capabilities of the model in neuroimage segmentation tasks. Our pretrained models and code are at https://github.com/lab-smile/BrainSegFounder. • We build a multi-modal 3D foundation model for neuroimaging segmentation • We propose a novel self-supervised two-stage pretraining scheme for medical AI • Our model outperforms supervised learning and traditional pretraining in segmentation • We show pretraining on healthy data improves model performance on diseased cases [ABSTRACT FROM AUTHOR]

In neuro-oncology, the precise segmentation of brain tumors from Magnetic Resonance Images is crucial for diagnosis, treatment planning, and monitoring disease progression. Accurate segmentation helps determine the tumor's size, location, and growth potential, which is essential for formulating effective treatment strategies. In response to this challenge, we developed a novel approach using Chaotic Local Search-Enhanced Differential Evolution (CJADE). CJADE, particularly its variant CJADE-M, which employs chaotic maps selected through a probability-based approach, has proven effective in optimizing brain tumor segmentation. Our study shows that CJADE-M outperforms traditional metaheuristic algorithms on various evaluation metrics. We further enhanced CJADE-M with an entropy-based hybrid objective function, which improved accuracy and reduced computational time in tumor segmentation compared to conventional methods like Minimum Cross-Entropy and Kapur. This makes our method suitable for real-time medical imaging analysis. Our findings indicate that CJADE-M, equipped with the hybrid objective function, achieves superior segmentation performance for both benign lobulated and malignant irregular tumors across metrics such as PSNR, FSIM, QILV, and HPSI. By providing a more accurate and efficient tool, our approach can significantly enhance the outcomes of brain tumor diagnosis and treatment, improving patient care in neuro-oncology. • CJADE algorithm first considered for Brain Tumor Segmentation (BTS). • A study to determine which CJADE variant is superior for BTS. • Best-performing CJADE-M variant compared with other metaheuristic algorithms. • Enhanced CJADE-M with a hybrid entropy-based objective function introduced. • This function with CJADE-M outperforms traditional objective functions. [ABSTRACT FROM AUTHOR]

Accurate and reliable segmentation of brain tumors from MRI images helps in planning an enhanced treatment and increases the life expectancy of patients. However, the manual segmentation of brain tumors is subjective and more prone to errors. Nonetheless, the recent advances in convolutional neural network (CNN)-based methods have exhibited outstanding potential in robust segmentation of brain tumors. This article comprehensively investigates recent advances in CNN-based methods for automatic segmentation of brain tumors from MRI images. It examines popular deep learning (DL) libraries/tools for an expeditious and effortless implementation of CNN models. Furthermore, a critical assessment of current DL architectures is delineated along with the scope of improvement. In this work, more than 50 scientific papers from 2014-2020 are selected using Google Scholar and PubMed. Also, the leading journals related to our work along with proceedings from major conferences such as MICCAI, MIUA and ECCV are retrieved. This research investigated various annual challenges too related to this work including Multimodal Brain Tumor Segmentation Challenge (MICCAI BRATS) and Ischemic Stroke Lesion Segmentation Challenge (ISLES). After a systematic literature search pertinent to the theme, we found that principally there exist three variations of CNN architecture for brain tumor segmentation: single-path and multi-path, fully convolutional, and cascaded CNNs. The respective performances of most automated methods based on CNN are appraised on the BraTS dataset, provided as a part of the MICCAI Multimodal Brain Tumor Segmentation challenge held annually since 2012. Notwithstanding the remarkable potential of CNN-based methods, reliable and robust segmentation of brain tumors continues to be an intractable challenge. This is due to the intricate anatomy of the brain, variability in its appearance, and imperfection in image acquisition. Moreover, owing to the small size of MRI datasets, CNN-based methods cannot operate with their full capacity, as demonstrated with large scale datasets, such as ImageNet. [ABSTRACT FROM AUTHOR]

• An accurate and efficient model for fully automatic brain tumor segmentation. • A novel CNN architecture, ESA-Net, is proposed as an improved variant of U-Net. • A new composite loss function is proposed for segmentation accuracy improvement. • Our method achieved competitive accuracy compared to state-of-the-art methods. • Our method is over 150 times faster and smaller than other state-of-the-art methods. Automatically segmenting tumors from brain magnetic resonance imaging scans is crucial for diagnosis and planning treatment. However, brain tumors are highly diverse in location, contrast, size, and shape, making automatic segmentation extremely challenging. Recent techniques for segmenting brain tumors are mostly built using convolutional neural networks (CNNs). However, most of these existing techniques are inefficient, having slow inference speed and high parameter count. To reduce the diagnostic time, we present an accurate and efficient CNN model having fast inference speed and low parameter count for fully automatic brain tumor segmentation. Our novel CNN, the efficient spatial attention network (ESA-Net), is an improved variant of the popular U-Net. ESA-Net was built using our proposed efficient spatial attention (ESA) blocks containing depthwise separable convolution layers and a lightweight spatial attention module. The ESA blocks significantly improve both efficiency and segmentation accuracy. We also proposed a new composite loss function by combining Dice, focal, and Hausdorff distance (HD) losses to significantly improve the segmentation accuracy by tackling extreme class imbalance and directly optimizing the Dice score and HD. The effectiveness of the proposed network and loss function was evaluated by performing extensive experiments on the BraTS 2021 benchmark dataset. ESA-Net significantly outperformed U-Net in segmentation accuracy while having four times faster inference speed and eight times fewer parameters. In addition, the composite loss outperformed other loss functions. The proposed model achieved significantly better segmentation accuracy than other efficient models while having faster inference speed and fewer parameters. Moreover, it obtained competitive segmentation accuracy against state-of-the-art models. The proposed system segments a patient's brain in 2.7 s using a GPU and has 157 times faster inference speed and 177 times fewer parameters than other state-of-the-art systems. [ABSTRACT FROM AUTHOR]

Brain tumor segmentation technology is crucial in diagnosing and treating MRI brain tumors. It aids doctors in locating and measuring tumors. It also plays a vital role in the planning and development of treatment and rehabilitation strategies. Recently, challenges have persisted despite the success achieved by deep learning-based methods in improving the segmentation accuracy of MRI brain tumor regions. These challenges are particularly prominent in small-scale tumor regions, where issues arise due to their diminutive size and the substantial variation between regions occupied by different tumor classes. Additionally, limitations in parameters and computational complexity contribute to these challenges. Thus, there remains considerable potential for enhancing these methods and increasing their efficacy. This paper presents an effective 2D residual neural network (2D ERU-Net) for segmenting MRI brain tumors. The proposed architecture consists of residual units in the encoder path to speed up training and convergence, and a deep supervision module (DSM) in the decoder path to address gradient-related issues and obtain high-resolution feature maps. Furthermore, a fusion weighted loss function, incorporating both weighted dice loss and weighted cross-entropy loss, is developed to solve the issues related to network convergence and data imbalance. The results of experiments on the reliable MRI brain tumor BraTS2019 dataset showed that the ERU-Net achieved a mean dice coefficient of 0.881, 0.882, and 0.891 and Hausdorff distances of 20.86 mm, 17.83 mm, and 13.35 mm for the tumor core, enhancing tumor, and whole tumor, respectively. Furthermore, ERU-Net demonstrates exceptional efficiency when compared to state-of-the-art methods. • This paper presents a 2D residual network for brain tumor segmentation from MRI. • ERU-Net combines the efficiency and the residual connections resilience. • We used BraTS 2019 dataset to perform brain tumor segmentation. • The proposed model contains residual units and the deep supervision module (DSM). • We used a fusion weighted loss to solve the issues of convergence and data imbalance. [ABSTRACT FROM AUTHOR]

Overall subdivision flow chart of CDSG-SAM To solve the cross-domain few-shot brain tumor segmentation problem. We propose CDSG-SAM, a cross-domain self-generating prompt few-shot brain tumor segmentation pipeline for fast few-shot brain tumor segmentation. The pipeline is based on the Segment Anything model (SAM), which integrates a novel cross-domain Self-attention (CDS) adapter module and a Self-generating (SG) prompting module. Sam-based segmentation model is more suitable for small sample medical image segmentation. In the decoder stage, a dynamic blur support Mask decoder module (DFSMD) is proposed to enhance the accuracy of brain tumor segmentation edges. The joint loss function (JL) is designed to comprehensively consider multiple aspects of performance indicators to improve the accuracy and robustness of the model. [Display omitted] • A new Cross-domain Self-attention (CDS) Adapter encoder adapter and self- generating Prompt (SG prompt) module are proposed. The CDS Adapter is used to solve the cross-domain multi-scale few-shot information extraction. The SG Prompt module was creatively proposed to generate an auxiliary prompt mask to solve the limitation of manual prompting in SAM. • A dynamic fuzzy support mask decoder (DFSMD) module and joint loss function (JL) are proposed. DFSMD module is proposed to enhance the accuracy of brain tumor segmentation edges. Joint loss function is designed to comprehensively consider multiple aspects of performance metrics. • Validate d the effectiveness of CDSG-SAM on multiple datasets. To evaluate the generalizability of our proposed segmentation method across various datasets, we have obtained promising results from both publicly accessible and clinically collected datasets. In clinical practice, accurate segmentation of brain tumor regions is essential for patient treatment and survival. Accurate brain tumor segmentation is an important task and it is based on a large amount of labeled data. Clinical data are from different cities and device protocols, which leads to the poor generalization ability of the model. In addition, the small amount of clinical data also leads to the decline of the segmentation performance. In this paper, a cross-domain self-generating prompt few-shot brain tumor segmentation pipeline called CDSG-SAM is developed. This pipeline is based on the Segment Anything Model (SAM), which integrates a new Cross-domain self-attention (CDS) Adapter module and Self-Generating (SG) Prompt module. The SAM-based segmentation model is more suitable for few-shot medical image segmentation. A dynamic fuzzy support mask decoder module (DFSMD) is proposed in the decoder stage to enhance the accuracy of brain tumor segmentation edges. Furthermore, a Joint Loss function (JL) is designed to comprehensively consider multiple aspects of performance indicators to improve the accuracy and robustness of the model. Experimental results evaluated on the BraTS2021 glioma dataset, Clinical glioma brain tumor data (CBTD), and BraTS2023 metastatic tumor dataset show that the proposed pipeline has achieved excellent performance with an average dice coefficient of 0.918, 0.828, and 0.868 which outperforms than existing methods. [ABSTRACT FROM AUTHOR]

Accurate brain tumor image segmentation can significantly assist doctors in clinical diagnosis and treatment. The complementary information from multimodal magnetic resonance imaging provides more comprehensive information on brain tumors for lesion segmentation. However, the complexity of clinical scenarios may lead to modal loss or damage, thereby reducing segmentation performance. Therefore, brain tumor image segmentation becomes a challenging task in the absence of modalities. In recent years, leveraging interactive mechanisms to learn shared feature representations across multiple modalities has become a dominant approach, especially for enhancing segmentation accuracy in scenarios with incomplete modalities. However, a significant challenge in most existing methods is their inability to effectively capture complete multimodal information due to the loss of modality-specific details during interaction processes. To overcome this challenge, we introduce a novel network, referred to as the Multimodal Invariant Feature Prompt Network (MIFPN), which enhances learning by incorporating modality prompts within multimodal interactions, thus ensuring a more comprehensive acquisition of modality information. This network aims to introduce modality-invariant feature prompts in modal information interaction, guiding the model to learn the missing modality information and facilitating integration. To fully leverage modal information, we learned both modality-invariant and modality-specific information, and fused them during the prompt learning process to obtain a more comprehensive understanding of modal information. Considering the different contributions of various modalities to the segmentation results, we designed modality-aware masks and modality selection strategies to merge shallow features of the encoder. MIFPN effectively mitigates the brain tumor image segmentation problem under missing modalities by learning from missing modality prompts and modality-aware fusion. Extensive experiments conducted on the BraTS2020 and BraTS2018 datasets show that our method outperforms state-of-the-art methods in most scenarios across 15 missing modality configurations. The code is available at https://github.com/diaoyq121/MIFPN. [ABSTRACT FROM AUTHOR]

Accurate segmentation of brain tumors across multiple MRI modalities is crucial for clinical diagnosis and prognosis. However, encountering missing modalities due to practical factors is common. Recent years have witnessed the emergence of various approaches, including 3D convolutional and transformer-based models, showcasing impressive performance in addressing this challenge. Nonetheless, these methods often suffer from high GPU memory requirements during inference, limiting their applicability. To explore alternative solutions, we turned to 2D-convolution-based techniques. Generative learning-based methods have produced visually appealing reconstructed MRI images, yet few consider the gap between visualization and segmentation. Drawing inspiration from clinicians who leverage cross-modality information for tumor estimation, we adopt a generative modality reconstruction task for pre-training, followed by fine-tuning for segmentation on MRI data with missing modalities. Furthermore, our investigation revealed common feature representations shared between reconstruction and segmentation tasks in latent space, as demonstrated through feature distributions. By utilizing a multi-encoder Unet architecture with fine-tuned adjustments, our method not only attains competitive performance in segmentation but also markedly decreases GPU memory usage during inference. • A new framework is proposed for brain tumor segmentation with missing modalities. • Our framework achieve an optimal balance between accuracy and computation cost. • We demonstrate how the generative reconstruction stage enhances the segmentation stage. [ABSTRACT FROM AUTHOR]

• This study proposes a lightweight automatic 3D algorithm with a 3D attention mechanism. • The segmentation of brain tumor images addresses the practical challenges. • Existing attention mechanisms lack the ability to effectively utilize the spatial information of brain tumor images. • The results of the proposed network model, applied to the BraTS 2019 test set. The Dice coefficients for enhancing tumor, whole tumor and tumor core are 79.07 %, 90.21 % and 84.09 %, and the proposed network model was achieved using 0.19 M parameters and 17.96 FLOPs. Accurate identification and segmentation of brain tumors from multimodal MRI images is essential for making treatment decisions and planning surgeries. However, the complexity associated with many neural network deployments presents challenges in meeting specific clinical accuracy requirements. In order to tackle these challenges, this paper presents a segmentation algorithm that is lightweight and efficient. Within the framework of 3D U-Net, the following enhancements are proposed: (1) in this study, we replace standard convolution with hierarchical decoupled convolution and employing trilinear interpolation for up-sampling in order to reduce model complexity; (2) in this study, based on the hierarchical decoupled convolution modules, we have also designed partially hierarchical decoupled convolution units to enhance the network's ability to capture multi-scale information; (3) in this study, we introduce a 3D attention mechanism to facilitate comprehensive learning of three-dimensional information in brain tumor images, thereby enhancing the segmentation accuracy of both brain tumors and their adjacent regions. The BraTS series of datasets represents a commonly used and authoritative public resource for brain tumor image segmentation. After implementing the TDPC-Net algorithm on the BraTS 2019 validation set, we achieved Dice scores of 79.07 %, 90.21 % and 84.09 % for enhancing tumor, whole tumor and tumor core, respectively. These results were attained using 0.19 million parameters and 17.96 giga floating-point operations. Our proposed model has undergone extensive testing on the BraTS 2018, BraTS 2019 and BraTS 2020 datasets, demonstrating its exceptional capability in accurately segmenting brain tumor regions. You can find the code online at: https://github.com/liyixuansust/TDPC-NET. [ABSTRACT FROM AUTHOR]

Automatic segmentation of brain tumors employing images from multi-modalities is important for preoperative diagnosis and prognostic assessment. The rich complementary information contained within multi-modal images allows for improved brain tumor segmentation performance when models are trained on multi-modal data. However, accurate segmentation of small lesion regions from medical images remains challenging, due to the irregular shapes and low boundary contrast of brain gliomas. To address these challenges, we propose a coarse-to-fine feature fusion network (CFNet) that effectively incorporates multi-modal image features through modal interaction, semantic perception, and feature fusion. Specifically, the modality cross-attention fusion module is proposed introducing complementary features and mapping them into a unified semantic space to learn complementary representations between modalities. Furthermore, the multi-scale context perception module enables the model to focus on multi-scale lesion fine-grained information while capturing deep semantics. Moreover, the further fusion of fine-grained features with coarse-grained features enhances edge features and the complementary of features. We evaluate the proposed CFNet on two public brain tumor segmentation datasets BraTS2019 and BraTS2020. Experimental results show that the proposed framework outperforms state-of-the-art methods in Dice score, HD95, Sensitivity, demonstrating the effectiveness of CFNet on brain tumor segmentation. Code will be available at https://github.com/YaruC/CFNet. • A multi-modal segmentation model is proposed for the segmentation of brain tumor. • Introducing a cross-attention mechanism to facilitate semantic alignment in images. • Enhancing multi-scale lesion features through multi-level perception. • Integrating coarse-grained and fine-grained features to enhance edge features. characterization. [ABSTRACT FROM AUTHOR]

Brain tumor, abnormal cell growth within the brain, require precise segmentation to facilitate effective treatment planning. Accurately identifying tumor boundaries from complex Magnetic Resonance Imaging (MRI) scans is critical to this process. However, this task poses significant challenges due to the intricate and variable nature of brain tumors. Attention-enhancing mechanisms are integrated to refine segmentation performance by capturing crucial spatial, channel-wise, and contextual features. This study proposes an innovative cum hybrid approach combining transfer and self-supervised learning to improve brain tumor segmentation. Obtaining annotated medical images for segmentation tasks in healthcare is challenging due to privacy concerns, limited availability, the time-consuming nature of manual annotation, and the need for expert knowledge in the domain. The proposed approach enhances the segmentation process's efficiency by tapping into prior knowledge from medical imaging data and allowing the model to learn from unannotated data. A specialized 3D UNET architecture is employed for fine-tuning with limited labeled data. Notably, this technique reduces the reliance on extensive annotated data, making it computationally feasible and potentially valuable for clinical use. Incorporating transfer learning and self-supervised learning provides a robust and accurate solution for brain tumor segmentation, achieving a high dice similarity coefficient of 90.15. This paper underscores the potential of advanced learning techniques in addressing the challenges of brain tumor segmentation in 3D MRI scans. [ABSTRACT FROM AUTHOR]

Brain tumor segmentation is currently of a priori guiding significance in medical research and clinical diagnosis. Brain tumor segmentation techniques can accurately partition different tumor areas on multi-modality images captured by magnetic resonance imaging (MRI). Due to the unpredictable pathological process of brain tumor generation and growth, brain tumor images often show irregular shapes and uneven internal gray levels. Existing neural network-based segmentation methods with an encoding/decoding structure can perform image segmentation to some extent. However, they ignore issues such as differences in multi-modality information, loss of spatial information, and under-utilization of boundary information, thereby limiting the further improvement of segmentation accuracy. This paper proposes a multimodal spatial information enhancement and boundary shape correction method consisting of a modality information extraction (MIE) module, a spatial information enhancement (SIE) module, and a boundary shape correction (BSC) module. The above three modules act on the input, backbone, and loss functions of deep convolutional networks (DCNN), respectively, and compose an end-to-end 3D brain tumor segmentation model. The three proposed modules can solve the low utilization rate of effective modality information, the insufficient spatial information acquisition ability, and the improper segmentation of key boundary positions can be solved. The proposed method was validated on BraTS2017, 2018, and 2019 datasets. Comparative experimental results confirmed the effectiveness and superiority of the proposed method over state-of-the-art segmentation methods. • Proposes a novel CNN-based brain tumor segmentation method. • Introduces an MIE module to utilize the multi-modality information better. • Designs an SIE module to preserve the spatial information better. • Presents a BSC module to improve the segmentation accuracy of boundaries. • Achieves the state-of-the-art brain tumor segmentation results. [ABSTRACT FROM AUTHOR]

Accurate brain tumor segmentation in MRI images is crucial for effective treatment planning and monitoring. Traditional methods often encounter challenges due to the complexity and variability of tumor shapes and textures. Consequently, there is a growing need for automated solutions to assist healthcare professionals in segmentation tasks, improving efficiency and reducing workload. This study introduces an innovative method for accurately segmenting brain tumors in MRI images by employing a refined 3D UNet model integrated with a Transformer. The goal is to leverage self-attention mechanisms to enhance segmentation capabilities. The proposed model combines Contextual Transformer (CoT) and Double Attention (DA) architectures. CoT is extended to a 3D format and integrated with the baseline model to exploit intricate contextual details in MRI images. DA blocks in skip connections aggregate and distribute long-range features, emphasizing inter-dependencies within an expanded spatial scope. Experimental results demonstrate superior segmentation performance compared to current state-of-the-art methods. With its ability to accurately segment and delineate tumors in 3D, our segmentation model promises to be a powerful tool for medical image processing and performance optimization, saving time for healthcare professionals and healthcare systems. [ABSTRACT FROM AUTHOR]

Despite the great success of deep neural networks in brain tumor segmentation, it is challenging to obtain sufficient annotated images due to the requirement of clinical expertise. Masked image modeling recently achieved competitive performance compared with supervised training by learning rich representations from unlabeled data. However, it is originally designed for vision transformers and its effectiveness has not been well-studied in the medical domain, usually for limited unlabeled data and small convolutional network scenarios. In this paper, we propose a self-supervised learning framework to pre-train U-Net for brain tumor segmentation. Our goal is to learn modality-specific and modality-invariant representations from multi-modality magnetic resonance images. This is motivated by the fact that different modalities indicate the same organs and tissues but have various appearances. To achieve this, we design a new pretext task that reconstructs the masked patches of each modality based on the partial observation of other modalities. We evaluate our method by transfer performance on BraTS 2020 dataset. The experimental results demonstrate our method outperforms other self-supervised learning methods and improves the performance of a strong fully supervised baseline. The source codes are available at https://github.com/mobiletomb/IS-MIM. • We applied masked image modeling to pre-train brain tumor segmentation models. • A novel pretext task is proposed to improve pre-train data efficiency. • The proposed method is effective in limited pre-training datasets and small networks. • The fine-tuned model could improve the segmentation performance. [ABSTRACT FROM AUTHOR]

Accurate brain tumor segmentation is a labor-intensive and time-consuming task that requires automation to enhance its efficacy. Recent advanced techniques have shown promising results in segmenting brain tumors; however, their dependency on extensive multimodal magnetic resonance imaging (MRI) data limits their practicality in clinical environments where such data may not be readily available. To address this, we propose a novel multi-teacher cross-modal knowledge distillation framework, which utilizes the privileged multimodal data during training while relying solely on unimodal data for inference. Our framework is tailored to the unimodal segmentation of the T 1ce MRI sequence, which is prevalently available in clinical practice and structurally akin to the T 1 modality, providing ample information for the segmentation task. Our framework introduces two learning strategies for knowledge distillation (KD): (1) performance-aware response-based KD and (2) cooperative deep supervision fusion learning (CDSFL). The first strategy involves dynamically assigning confidence weights to each teacher model based on its performance, ensuring that the KD is performance-driven, and the CDSFL module augments the learning capabilities of the multi-teacher models by fostering mutual learning. Moreover, the fused information is distilled into the student model to improve its deep supervision. Extensive experiments on BraTS datasets demonstrate that our framework achieves promising unimodal segmentation results on the T 1ce and T 1 modalities and outperforms previous state-of-the-art methods. Code is available at https://github.com/ami-lab-knut/mtcm_kd. [ABSTRACT FROM AUTHOR]

Brain tumor segmentation is an essential issue in medical image segmentation. However, it is still challenging to consider the relationship between modules and efficiently fuse the features between adjacent scales. In this paper, we propose a novel cross-attention network for brain tumor segmentation, namely multi-scales and multi-modules cross-attention UNet (MM-UNet), which contains two mechanisms, module and scale cross-attentions. The module cross-attention (M C A) strategy connects and exchanges global information between adjacent modules. The scale cross-attention (S C A) strategy has two policies, the scale-related non-local relationship module (S C A S N R ) and the scale-related channel-based relationship module (S C A S C R ), that fuses the information between adjacent scales to mix the multi-scale information. Experiments on well-known tumor datasets, BraTS 2020, which has 369 cases, and has been classified into training, validation, and testing sets with 17,576, 4395, and 5735 images, to evaluate the performance by segmenting three regions, the whole tumor area (WT), core tumor area (CT) and enhancing tumor area (ET). Moreover, we consider three numerical metrics, dice, precision, and Hausdorff metrics, and various visualization results to objectively evaluate and intuitively display the experimental results. The proposed model surpasses state-of-the-art methods and achieves 0.8519, 0.8889, and 1.2647 with a base version network in dice, precision, sensitivity, and Hausdorff metrics, respectively. Moreover, we demonstrate the visualization with segmentation results and heatmaps in various scenarios to present the robustness of the proposed network in each region. [Display omitted] • M C A fuses multimodalitites using global and long-range contextual interaction. • S C A (S C A S N R + S C A S C R ) fuses the global information between adjacent scales. • S C A S N R constructs the scale-related weighted long-distance dependencies. • S C A S C R generates the channel-based scale-related matrix, enhancing the features. • MM-UNet improves self-attention learning and performance on well-known benchmarks. [ABSTRACT FROM AUTHOR]

Deep learning technologies have led to substantial breakthroughs in the field of biomedical image analysis. Accurate brain tumor segmentation is an essential aspect of treatment planning. Radiologists agree that manual segmentation is a difficult and time-consuming task that frequently causes delays in the diagnosing process. While U-Net-based methods have been widely used for brain tumor segmentation, many challenges persist, particularly when dealing with tumors of varying sizes, locations, and shapes. Additionally, segmenting tumor regions with structures requires a comprehensive model, which can increase computational complexity and potentially cause gradient vanishing issues. This study presents a novel method called 3D Guided Attention-based deep Inception Residual U-Net (GAIR-U-Net) to address these challenges. This model combines attention mechanisms, an inception module, and residual blocks with dilated convolution to enhance feature representation and spatial context understanding. The backbone of the model is the U-Net model, which leverages the power of inception and residual connections to capture intricate patterns and hierarchical features while expanding the model's width in three-dimensional space without significantly increasing computational complexity. The attention mechanisms play a role in focusing on important regions and areas while downgrading irrelevant details. The dilated convolutions in the network help in learning both local and global information, improving accuracy and adaptability in segmenting tumors. All the experiments in this study were carried out on multimodal MRI scans that include (T1-weighted, T1-ce, T2-weighted, and FLAIR sequences) from the BraTS 2020 dataset. The presented model is trained and tested on the same dataset, which exhibited promising performance compared to previous methods. On the BraTS 2020 validation dataset, the proposed model obtained a dice score of 0.8796, 0.8634, and 0.8441 for whole tumor (WT), tumor core (TC), and enhancing tumor (ET), respectively. These results demonstrate the model's efficacy in precisely segmenting brain tumors across various modalities. Comparative analyses underscore the model's versatility in handling tumor shape variations, size, and location, making it a promising solution for clinical applications. [ABSTRACT FROM AUTHOR]

Atrous convolution and attention have improved the performance of the UNet architecture for segmentation purposes. However, a perfect combination of atrous convolution and attention to improve brain tumor segmentation performance is still an interesting challenge. In this paper, we propose UNet architecture with the addition of attention in the skip connection and the replacement of the processing block with two atrous convolution sequences connected to the attention unit combined with one residual path called the Multipath Residual Attention Block (MRAB). The architecture was trained using the Brain Tumor Segmentation(BraTS) 2018, 2019, 2020, and 2021 challenge datasets. The ensembled model was validated online and obtained dice scores of 77.71%, 79.77%, 89.59% for BraTS2018, 74.91%, 80.98%, 88.48% for BraTS2019, 72.91%, 80.19%, 88.57% for BraTS2020, and 77.73%, 82.19%, 89.33% for BraTS2021 validation datasets for Enhanced Tumor(ET), Tumor Core(TC), and Whole Tumor(WT) areas, respectively. These dice score performances outperformed state-of-the-art brain tumor segmentation architectures and promised to be developed for clinical application. [ABSTRACT FROM AUTHOR]

• A conditional generator is designed to generate the missing modality with correlation constraint among the modalities. • A multi-source correlation network is introduced to learn the latent multi-source correlation among modalities. • A brain tumor segmentation network via multi-source correlation to conditionally generate the missing modality is proposed. • The experimental results demonstrate the effectiveness of our proposed method. Brain tumor is one of the most high-risk cancers which causes the 5-year survival rate of only about 36%. Accurate diagnosis of brain tumor is critical for the treatment planning. However, it's common to missing one modality in clinical scenarios. In this paper, we propose a novel brain tumor segmentation network to impute the missing data. The proposed network consists of a conditional generator, a multi-source correlation network and a segmentation network. To impute the missing data, we propose to use a conditional generator to generate the missing modality under the condition of the available modalities. As the multi MR modalities have a strong relationship in tumor regions, we design a multi-source correlation network to learn the multi-source correlation. On the one hand, the multi-source correlation network can help the conditional generator to generate the missing modality which should keep the consistent correlation with the available modalities. On the other hand, it can guide the segmentation network to learn the correlated feature representations to improve the segmentation performance. The experiments evaluated on BraTS 2018 dataset demonstrate the superior performance of the proposed method when compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Using multimodal Magnetic Resonance Imaging (MRI) is necessary for accurate brain tumor segmentation. The main problem is that not all types of MRIs are always available in clinical exams. Based on the fact that there is a strong correlation between MR modalities of the same patient, in this work, we propose a novel brain tumor segmentation network in the case of missing one or more modalities. The proposed network consists of three sub-networks: a feature-enhanced generator, a correlation constraint block and a segmentation network. The feature-enhanced generator utilizes the available modalities to generate 3D feature-enhanced image representing the missing modality. The correlation constraint block can exploit the multi-source correlation between the modalities and also constrain the generator to synthesize a feature-enhanced modality which must have a coherent correlation with the available modalities. The segmentation network is a multi-encoder based U-Net to achieve the final brain tumor segmentation. The proposed method is evaluated on BraTS 2018 dataset. Experimental results demonstrate the effectiveness of the proposed method which achieves the average Dice Score of 82.9, 74.9 and 59.1 on whole tumor, tumor core and enhancing tumor, respectively across all the situations, and outperforms the best method by 3.5%, 17% and 18.2%. [ABSTRACT FROM AUTHOR]

Automatic segmentation of brain tumor from Magnetic Resonance Images (MRI) is one of the challenging tasks in computer vision. Many proposals investigate the use of Deep Neural Networks (DNN) in image segmentation as they have a high performance in automatic segmentation of brain tumors images. Due to the gradient diffusion problem and complexity, it generally takes a lot of time and extra computational power for training deeper neural networks. In this paper, we present an automatic technique for brain tumor segmentation depending on Deep Residual Learning Network (ResNet) to get over the gradient problem of DNN. ResNets accomplish more accuracy and can make the training process faster compared to their equivalent DNN. To achieve this enhancement, ResNets add a shortcut skip connection parallel to convolutional neural networks layers. Simulation examples have been carried out on dataset BRATS 2015 to verify the superiority of the proposed technique. Results verify that the proposed technique has an improved accuracy of 83%, 90%, and 85% for the complete, core, and enhancing regions, respectively. Moreover, it has an average computation time (3 times) faster than other DNN techniques. [ABSTRACT FROM AUTHOR]

Brain tumors are threatening the life and health of people in the world. Automatic brain tumor segmentation using multiple MR images is challenging in medical image analysis. It is known that accurate segmentation relies on effective feature learning. Existing methods address the multi-modal MR brain tumor segmentation by explicitly learning a shared feature representation. However, these methods fail to capture the relationship between MR modalities and the feature correlation between different target tumor regions. In this paper, I propose a multi-modal brain tumor segmentation network via disentangled representation learning and region-aware contrastive learning. Specifically, a feature fusion module is first designed to learn the valuable multi-modal feature representation. Subsequently, a novel disentangled representation learning is proposed to decouple the fused feature representation into multiple factors corresponding to the target tumor regions. Furthermore, contrastive learning is presented to help the network extract tumor region-related feature representations. Finally, the segmentation results are obtained using the segmentation decoders. Quantitative and qualitative experiments conducted on the public datasets, BraTS 2018 and BraTS 2019, justify the importance of the proposed strategies, and the proposed approach can achieve better performance than other state-of-the-art approaches. In addition, the proposed strategies can be extended to other deep neural networks. • A novel multi-modal brain tumor segmentation network is proposed. • A multi-modal fusion module is introduced to fuse multi-modalities. • Disentangled representation learning is proposed to learn tumor-related features. • Region-aware contrastive learning is designed to learn useful feature representations. • Extensive experiments demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

• A lightweight architecture is proposed for the segmentation of brain tumor regions. • Proposed multi-perspective extraction (MPE) module allows for improving the relevance of contextual feature extraction. • Dense attention (DE) module is designed to provide each point with an appropriate level of attention while fusing features. • A novel loss function of segmentation and special data preprocessing are explored in this work. Malignant brain tumors are highly deadly, necessitating the quickly precise segmentation of tumor regions. Previously, clinicians manually classified brain tumor regions utilizing magnetic resonance imaging (MRI). A new trend is the use of computer vision processing to assist clinicians in clinical analysis. Even though numerous recent methodologies based on CNN have been presented, there remains a lack of high performance when evaluating regions in MRI images. Furthermore, there is still a possibility for development in terms of parameter number and computational complexity. To collect feature information and improve the relevance of contextual feature extraction, a multi-perspective extraction (MPE) module is proposed. MPE consists of three different convolutional kernels and special operations. In addition, a dense attention (DA) module is used to provide each point with an appropriate level of attention while fusing features. The effectiveness of these two modules has been proved through ablation experiments. The proposed MPEDA-Net achieves dice of 82.52%, 93.07%, and 87.67% on the R-BraTS2021 (reconstructed-BraTS2021) in the ET, WT, and TC respectively. In addition, the BraTS2018 and BraTS2019 experiments illustrate that the dice of ET, WT, and TC reach 82.44%, 91.38%, 88.27%, and 83.73%, 91.87%, 88.71%, respectively. The more effective segmentation performance shows that MPEDA-Net can significantly enhance brain tumor segmentation accuracy, exceeding several existing methods. The MPEDA-Net code is already available on GitHub: https://github.com/luohaohaoluo/MPEDANet-pytorch. [ABSTRACT FROM AUTHOR]

Multimodal magnetic resonance imaging (MRI) provides complementary information for brain tumor segmentation, and several methods leveraging full modalities have been proposed. However, capturing the full modality information is challenging due to commonplace data corruption, imperfect imaging protocols, and patient-related constraints. The unavailability of certain modalities can significantly undermine the performance of segmentation methods that rely on full-modality data. To address this issue, this paper proposes a deformation-aware and reconstruction-driven method for brain tumor segmentation in the presence of missing modalities. The proposed method introduces a local–global modeling module to enhance the intramodal feature representation ability of the modality-specific encoder. Considering the irregular shape of tumor regions, we develop a deformation-adaptive perceptual multimodal representation learning module that learns deformation information from an incomplete set of multimodal images, thereby guiding the network to accurately localize the tumor regions. Furthermore, we design a reconstruction-driven key-information mining module that recovers the original images from the features extracted by the encoder. This process further ensures that the encoder can extract the key tumor discriminative features. During the inference phase, the module is removed to mitigate additional computational burdens. Experimental results on two publicly available multimodal brain tumor benchmark datasets show that the proposed method outperforms existing brain tumor segmentation methods with missing modalities. The code is available at https://github.com/Linzy0227/SRMNet. • We propose a novel method for incomplete multimodal brain tumor segmentation. • We design a local and global modeling module extract modality-specific features. • We propose a deformable adaptive perception method for irregularly shaped tumor segmentation. [ABSTRACT FROM AUTHOR]

Accurate segmentation of brain tumors from multimodal MRI sequences is a critical prerequisite for brain tumor diagnosis, prognosis, and surgical treatment. While one or more modalities are often missing in clinical practice, which can collapse most previous methods that rely on all modality data. To deal with this problem, the current state-of-the-art Transformer-related approach directly fuses available modality-specific features to learn a shared latent representation, with the aim of extracting common features that are robust to any combinatorial subset of all modalities. However, it is not trivial to directly learn a shared latent representation due to the diversity of combinatorial subsets of all modalities. Furthermore, correlations across modalities as well as global multiscale features are not exploited in this Transformer-related approach. In this work, we propose a Multiscale Multimodal Vision Transformer (MMMViT), which not only leverages correlations across modalities to decouple the direct fusing procedure into two simple steps but also innovatively fuses local multiscale features as the input of the intra-modal Transformer block to implicitly obtain the global multiscale features to adapt to brain tumors of various sizes. We experiment on the BraTs 2018 dataset for all modalities and various missing-modalities as input, and the results demonstrate that the proposed method achieves the state-of-the-art performance. Code is available at: https://github.com/qiuchengjian/MMMViT. • Global multiscale features are implicitly extracted with limited computational cost. • Correlations across modalities are first used to decouple the segmentation pipeline. • Our method achieves the current state-of-the-art segmentation accuracy. [ABSTRACT FROM AUTHOR]

Brain tumor segmentation is one of the most challenging tasks for radiomics analysis. Manual segmentation by radiologists is both laborious and subjective, which limits the number of study cases and the reproductivity of each clinical study. Hence, the automatic tumor segmentation method is in high demand, not only to segment the tumor, but also divide it into several fine-grained specific categories in voxel level. To this end, we propose a 3D Center-crop Dense Block for medical images by introducing high-level supervision into the lower layers of neural networks and adopt a parallel pathway network architecture of attention and context pathways. The attention pathway is of normal resolution and mainly concentrate on the detailed information of each voxel, while the context pathway is of low resolution and focus on the surrounding information. Furthermore, we also involve cross-pathway connections from attention pathway to context pathway with weighted fusion structure to compensate the missing detailed information caused by downsampling. The segmentation performance on the BraTS 2015 and BraTS 2017 datasets corroborate the effectiveness of the proposed deep learning architecture over the state-of-the-art results. [ABSTRACT FROM AUTHOR]

Segmentation of multimodal brain tissues from 3D medical images is of great significance for brain diagnosis. It is required to create an automated and accurate segmentation based on deep-learning network due to the manual segmentation is time-consuming. However, how to segment medical images accurately and how to build neural network effectively with very limited computing resource, is still challenging task. To address these problems, we propose a novel model based on 3D fully convolutional network. More specifically, we apply multi-pathway architecture to feature extraction so as to effectively extract features from multi-modal MRI images. Different receptive field of feature have been extracted by adopting 3D dilated convolution in each pathway. By evaluating one-pathway model and key components of our model with a set of effective training schemes, we analyzed how these alternative methods affect the performance of experiments. Our proposed model was evaluated in the Brain Tumor Segmentation 2019 dataset (BraTS 2019), making an effective segmentation for the complete, core and enhancing tumor regions in Dice Similarity Coefficient metric (0.89, 0.78, 0.76) for the dataset. Also, we made a practice on BraTS 2018 using the same method with the Dice Similarity Coefficient metric of 0.90, 0.79, and 0.77 for the complete, core and enhancing tumor regions. Our method is inherently general and is a powerful tool to studies of medical images of brain tumors. Our code is available at https://github.com/JalexDooo/BrainstormTS. [ABSTRACT FROM AUTHOR]

Traditional deep convolutional neural networks for fully automatic brain tumor segmentation have two problems: spatial information loss caused by both the repeated pooling/striding and weak ability of multi-scale lesion processing. To overcome the first problem, we use a 3D atrous-convolution with a single stride to replace pooling/striding and build the backbone for feature learning. For the second problem, a 3D atrous-convolution feature pyramid is designed and added to the end of the backbone. By integrating with contextual features, this structure improves the discriminating ability of the overall model to segment tumors with various sizes. Finally, a 3D fully connected Conditional Random Field is constructed as a post-processing step for the network's output to obtain structural segmentation of both the appearance and spatial consistency. Abundant ablation experiments carried on Magnetic Resonance Imaging datasets demonstrate that lossless feature computation and multi-scale information fusion driven by our method are feasible to address the above problems. Compared with the state-of-the-art methods on the public benchmarks, our method achieves competitive performance and can be efficiently implemented into the clinical medical application. [ABSTRACT FROM AUTHOR]

Gliomas are the most frequent primary brain tumors, which have a high mortality. Surgery is the most commonly used treatment. Magnetic resonance imaging (MRI) is especially useful to assess gliomas and evaluate the success of the treatment in clinical practice. So accurately segmenting the brain tumor from MRI images is the key to clinical diagnostics and treatment planning. However, a large quantity of data produced by MRI prevents manual segmentation in a reasonable time. So automatic approaches are required for quick and effective segmentation. But the spatial and structural variability among brain tumors bring a challenge to automatically segment MRI images. Deep Convolutional Neural Network (DCNN) can achieve complex function mapping. This contributes to response to the challenge. Therefore, DCNN is the commonly used method in brain tumor segmentation. Although a large number of DCNN based methods have been proposed, these methods mainly aim to improve the quality of the image features extracted by DCNN. Actually, these methods usually ignore the prior knowledge in medical images, and the simple prior knowledge in brain tumor segmentation is that most of the tumor regions in images are left-right asymmetry. Based on this, in this paper, a novel deep convolutional neural network which combines symmetry have been proposed to automatically segment brain tumors. Our neural networks, called Deep Convolutional Symmetric Neural Network (DCSNN), extends DCNN based segmentation networks by adding symmetric masks in several layers. Our proposal was validated in the BRATS 2015 database, and we also give some baselines. The results are evaluated by dice similarity coefficient metric (DSC). And our proposed method achieved a competitive result with average DSC of 0.852. Furthermore, our method only takes about 10.8 s to segment a patient case. Although our method is not the best performance in BRATS 2015 challenge, as far as we know, our method outperforms other recent DCNN-based methods. [ABSTRACT FROM AUTHOR]

• Detailed discussion on the significance of MRI for brain tumor detection and classification of brain tumors. • Focus on Deep learning and metaheuristics methods for brain tumor classification through image segmentation. • Analysis of recent deep learning and metaheuristics methods from 2014 to 2019. The past few years have witnessed a significant increase in medical cases related to brain tumors, making it the 10th most common form of tumor affecting children and adults alike. However, it is also one of the most curable forms of tumors if detected well on time. Consequently scientists and researchers have been working towards developing sophisticated techniques and methods for identifying the form and stage of tumor. Magnetic Resonance Imaging (MRI) and Computer Tomography (CT) are two methods widely used for resectioning and examining the abnormalities in terms of shape, size or location of brain tissues which in turn help in detecting the tumors. MRI, due to its advantages over CT scan, discussed later in the paper, is preferred more by the doctors. The way towards sectioning tumor from MRI picture of a brain cerebrum is one of the profoundly engaged regions in the network of medical science as MRI is non-invasive imaging. This paper provides a systematic literature survey of techniques for brain tumor segmentation and classification of abnormality and normality from MRI images based on different methods including deep learning techniques, metaheuristic techniques and hybridization of these two. It includes presentation and quantitative investigation used in conventional segmentation and classification techniques. [ABSTRACT FROM AUTHOR]

In medical image processing, brain tumor detection and segmentation is a challenging and time-consuming task. Magnetic Resonance Image (MRI) scan analysis is a powerful tool in the recent technology that makes effective detection of the abnormal tissues from the brain. In the brain image, the size of a tumor can be varied for different patients along with the minute details of the tumor. It is a difficult task to diagnose and classify the tumor from numerous images for the radiologists. This paper developed a brain tumor classification using a hybrid deep autoencoder with a Bayesian fuzzy clustering-based segmentation approach. Initially, the pre-processing stage is performed using the non-local mean filter for denoising purposes. Then the BFC (Bayesian fuzzy clustering) approach is utilized for the segmentation of brain tumors. After segmentation, robust features such as, information-theoretic measures, scattering transform (ST) and wavelet packet Tsallis entropy (WPTE) methods are used for the feature extraction process. Finally, a hybrid scheme of the DAE (deep autoencoder) based JOA (Jaya optimization algorithm) with a softmax regression technique is utilized to classify the tumor part for the brain tumor classification process. The proposed scheme is implemented in a MATLAB environment. The simulation results are conducted by the BRATS 2015 database which proved that the proposed approach obtained the high classification accuracy (98.5 %) when compared to other state-of-art methods. [ABSTRACT FROM AUTHOR]

Accurate brain tumor segmentation is critical for diagnosis and treatment planning, whereby multi-modal magnetic resonance imaging (MRI) is typically used for analysis. However, obtaining all required sequences and expertly labeled data for training is challenging and can result in decreased quality of segmentation models developed through automated algorithms. In this work, we examine the possibility of employing a conditional generative adversarial network (GAN) approach for synthesizing multi-modal images to train deep learning-based neural networks aimed at high-grade glioma (HGG) segmentation. The proposed GAN is conditioned on auxiliary brain tissue and tumor segmentation masks, allowing us to attain better accuracy and control of tissue appearance during synthesis. To reduce the domain shift between synthetic and real MR images, we additionally adapt the low-frequency Fourier space components of synthetic data, reflecting the style of the image, to those of real data. We demonstrate the impact of Fourier domain adaptation (FDA) on the training of 3D segmentation networks and attain significant improvements in both the segmentation performance and prediction confidence. Similar outcomes are seen when such data is used as a training augmentation alongside the available real images. In fact, experiments on the BraTS2020 dataset reveal that models trained solely with synthetic data exhibit an improvement of up to 4% in Dice score when using FDA, while training with both real and FDA-processed synthetic data through augmentation results in an improvement of up to 5% in Dice compared to using real data alone. This study highlights the importance of considering image frequency in generative approaches for medical image synthesis and offers a promising approach to address data scarcity in medical imaging segmentation. • Utilizing a conditional GAN to synthesize multi-modal brain tumor images for deep learning-based segmentation networks. • Fourier domain adaptation (FDA) used to align synthetic and real images, reducing domain shift. • FDA demonstrates up to 5% improvement in segmentation performance and prediction confidence. • Study emphasizes the importance of considering image frequency in generative medical image synthesis approaches. [ABSTRACT FROM AUTHOR]

Deep learning methods have shown great potential in processing multi-modal Magnetic Resonance Imaging (MRI) data, enabling improved accuracy in brain tumor segmentation. However, the performance of these methods can suffer when dealing with incomplete modalities, which is a common issue in clinical practice. Existing solutions, such as missing modality synthesis, knowledge distillation, and architecture-based methods, suffer from drawbacks such as long training times, high model complexity, and poor scalability. This paper proposes IMS2Trans, a novel lightweight scalable Swin Transformer network by utilizing a single encoder to extract latent feature maps from all available modalities. This unified feature extraction process enables efficient information sharing and fusion among the modalities, resulting in efficiency without compromising segmentation performance even in the presence of missing modalities. Two datasets, BraTS 2018 and BraTS 2020, containing incomplete modalities for brain tumor segmentation are evaluated against popular benchmarks. On the BraTS 2018 dataset, our model achieved higher average Dice similarity coefficient (DSC) scores for the whole tumor, tumor core, and enhancing tumor regions (86.57, 75.67, and 58.28, respectively), in comparison with a state-of-the-art model, i.e. mmFormer (86.45, 75.51, and 57.79, respectively). Similarly, on the BraTS 2020 dataset, our model scored higher DSC scores in these three brain tumor regions (87.33, 79.09, and 62.11, respectively) compared to mmFormer (86.17, 78.34, and 60.36, respectively). We also conducted a Wilcoxon test on the experimental results, and the generated p -value confirmed that our model's performance was statistically significant. Moreover, our model exhibits significantly reduced complexity with only 4.47 M parameters, 121.89G FLOPs, and a model size of 77.13 MB, whereas mmFormer comprises 34.96 M parameters, 265.79 G FLOPs, and a model size of 559.74 MB. These indicate our model, being light-weighted with significantly reduced parameters, is still able to achieve better performance than a state-of-the-art model. By leveraging a single encoder for processing the available modalities, IMS2Trans offers notable scalability advantages over methods that rely on multiple encoders. This streamlined approach eliminates the need for maintaining separate encoders for each modality, resulting in a lightweight and scalable network architecture. The source code of IMS2Trans and the associated weights are both publicly available at https://github.com/hudscomdz/IMS2Trans. • We propose a scalable Swin Transformer for the incomplete MRI modalities brain tumor segmentation. • We introduce a lightweight MLP bottleneck that not only reduces model parameters but also obtains better feature maps. • We introduce a feature distillation to improve consistency across different modalities. • We propose a 3D multimodal CutMix specifically for multi-modal MRI to enhance the robustness. [ABSTRACT FROM AUTHOR]

• An U-shaped network (DAUnet) for brain tumor MRI image segmentation is proposed. • A BA module containing the Bottleneck module and 3D SC attention is designed. • Using 3D SC attention that introduces residual connection, the consistency of feature maps is guaranteed. • A CASP module is designed at the bottom of DAUnet to facilitate upsampling feature recovery. • The hidden layers of DAUnet are supervised using the supervised loss in deep supervision to make the model fit better. In MRI images, the brain tumor area varies greatly between individuals, and only relying on the judgment of clinicians is prone to misdiagnosis and misjudgment. Consequently, utilizing computer-aided diagnosis is of utmost significance in assisting clinicians with delineating the tumor region. Brain tumor MRI images are 3D images, and traditional segmentation methods tend to lose key information. Therefore, this paper proposes DAUnet, an U-shaped network for brain tumor MRI image segmentation combining deep supervision and convolutional attention. First, a module consisting of a Bottleneck module and attention (BA) module is designed. Here the attention not only uses spatial and channel (SC) attention but also adds residual connection, which is called 3D SC attention. Second, to enlarge the feature map receptive field without changing its resolution, a module consists of standard convolution and atrous spatial pyramid (CASP) module is designed. The feature map information is adjusted by standard convolution, subsequently, the feature map is provided as input to the ASP module. The CASP module fuses the features extracted by downsampling and performs upsampling operation, which strengthens the correlation between different layers of the network. Finally, using deep supervision as an auxiliary branch of the U-shaped network, it combines deep learning and regularization techniques to supervise the model during training, automatically finer parameters, and make the model fit better. Through experiments on BraTS 2020 and FeTS 2021 and comparison with other advanced methods, it has been demonstrated that DAUnet achieves precise segmentation of tumor regions in brain MRI images. [ABSTRACT FROM AUTHOR]

Image processing is a crucial effort to discover suspicious regions and robust features from magnetic resonance imaging (MRI). Many segmentation algorithms have been proposed to increase the accuracy of brain tumor detection. However, brain image segmentation is considered a complex and challenging step in medical image processing. K-means is an unsupervised learning method for MRI image clustering and other tasks such as image segmentation and pattern identification of interest. However, the K-means algorithm still has many shortcomings, including a low convergence rate, a tendency to get stuck in local minima, and sensitivity to initialization. An improved version of Aquila Optimizer (AO) boosted with an alternative selection scheme and Cauchy mutation, called Mutated Aquila Optimizer (MAO), and the K-Means algorithm were combined to create a new dynamic and intelligent clustering method for brain tumor segmentation. The proposed MAO-Kmeans approach aims to increase the ability of K-Means to automatically extract the correct number and location of cluster centers and the number of pixels in each cluster in abnormal MRI images (multiple sclerosis lesions). The experimental results on the Brain Tumor Segmentation dataset (BraTS 2020) demonstrated the effectiveness of the MAO-Kmeans method in improving the performance of conventional K-means clustering. Furthermore, qualitative and quantitative comparisons between MAO-Kmeans and the other 12 state-of-the-art segmentation techniques (which include machine learning, artificial neural network, and deep learning-based methods) demonstrate the superiority of our proposed algorithm. • This work proposed an MAO-Kmeans approach for the segmentation of brain tumors. • MAO is the AO variant embedded with Cauchy mutation and truncation selection schemes. • The initial performance of the MAO is tested and verified on CEC'21 benchmarks. • MAO-Kmeans approach is tested for brain tumor segmentation on BraTS 2020 dataset. • Quantitative and qualitative analysis shows the superiority of the MAO-Kmeans method. [ABSTRACT FROM AUTHOR]

This study proposes a deep convolutional neural network for the automatic segmentation of glioblastoma brain tumors, aiming sat replacing the manual segmentation method that is both time-consuming and labor-intensive. There are many challenges for automatic segmentation to finely segment sub-regions from multi-sequence magnetic resonance images because of the complexity and variability of glioblastomas, such as the loss of boundary information, misclassified regions, and subregion size. To overcome these challenges, this study introduces a spatial pyramid module and attention mechanism to the automatic segmentation algorithm, which focuses on multi-scale spatial details and context information. The proposed method has been tested in the public benchmarks BraTS 2018, BraTS 2019, BraTS 2020 and BraTS 2021 datasets. The Dice score on the enhanced tumor, whole tumor, and tumor core were respectively 79.90 %, 89.63 %, and 85.89 % on the BraTS 2018 dataset, respectively 77.14 %, 89.58 %, and 83.33 % on the BraTS 2019 dataset, and respectively 77.80 %, 90.04 %, and 83.18 % on the BraTS 2020 dataset, and respectively 83.48 %, 90.70 %, and 88.94 % on the BraTS 2021 dataset offering performance on par with that of state-of-the-art methods with only 1.90 M parameters. In addition, our approach significantly reduced the requirements for experimental equipment, and the average time taken to segment one case was only 1.48 s; these two benefits rendered the proposed network intensely competitive for clinical practice. • A deep convolutional neural network is for glioblastoma brain tumors' automatic segmentation. • A spatial pyramid module and attention mechanism are for the segmentation algorithm. • The proposed method is tested in the BraTS 2018 and BraTS 2019 validation datasets. • The Dice score on the enhanced tumor, whole tumor, and tumor core are tested. • Two benefits render the proposed network intensely competitive for clinical practice. [ABSTRACT FROM AUTHOR]

The Augmented Transformer U-Net (AugTransU-Net) is proposed to address limitations in existing transformer-related U-Net models for brain tumor segmentation. While previous models effectively capture long-range dependencies and global context, these works ignore the hierarchy to a certain degree and need more feature diversity as depth increases. The proposed AugTransU-Net integrates two advanced transformer modules into different positions within a U-shaped architecture to overcome these issues. The fundamental innovation lies in constructing improved augmentation transformer modules that incorporate Augmented Shortcuts into standard transformer blocks. These augmented modules are strategically placed at the bottleneck of the segmentation network, forming multi-head self-attention blocks and circulant projections, aiming to maintain feature diversity and enhance feature interaction and diversity. Furthermore, paired attention modules operate from low to high layers throughout the network, establishing long-range relationships in both spatial and channel dimensions. This allows each layer to comprehend the overall brain tumor structure and capture semantic information at critical locations. Experimental results demonstrate the effectiveness and competitiveness of AugTransU-Net in comparison to representative works. The model achieves Dice values of 89.7%/89.8%, 78.2%/78.6%, and 80.4%/81.9% for whole tumor (WT), enhancing tumor (ET) and tumor core (TC) segmentation on the BraTS2019-2020 validation datasets, respectively. The code for AugTransU-Net will be made publicly available at https://github.com/MuqinZ/AugTransUnet. [ABSTRACT FROM AUTHOR]

Abstract Brain tumor segmentation from Magnetic Resonance Imaging scans is vital for both the diagnosis and treatment of brain cancers. It is widely accepted that accurate segmentation depends on multi-level information. However, exiting deep architectures for brain tumor segmentation fail to explicitly encourage the models to learn high-quality hierarchical features. In this paper, we propose a series of approaches to enhance the quality of the learnt hierarchical features. Our contributions incorporate four aspects. First, we extend the popular DeepMedic model to Multi-Level DeepMedic to make use of multi-level information for more accurate segmentation. Second, we propose a novel dual-force training scheme to promote the quality of multi-level features learnt from deep models. It is a general training scheme and can be applied to many exiting architectures, e.g., DeepMedic and U-Net. Third, we design a label distribution-based loss function as an auxiliary classifier to encourage the high-level layers of deep models to learn more abstract information. Finally, we propose a novel Multi-Layer Perceptron-based post-processing approach to refine the prediction results of deep models. Extensive experiments are conducted on two most recent brain tumor segmentation datasets, i.e., BRATS 2017 and BRATS 2015 datasets. Results on the two databases indicate that the proposed approaches consistently promote the segmentation performance of the two popular deep models. [ABSTRACT FROM AUTHOR]

Highlights • A brain tumor segmentation method for FLAIR sequences MRI images. • Method is based on texture features and kernel sparse coding. • Avoid linear inseparability from the similarity between normal and non-normal tissues. Abstract An automatic brain tumor segmentation method based on texture feature and kernel sparse coding from FLAIR (fluid attenuated inversion recovery) contrast-enhanced MRIs (magnetic resonance imaging) is presented in this paper. First, the MRIs are pre-processed to reduce noise, enhance contrast and correct the intensity non-uniformity. Then sparse coding is performed on the first order and second order statistical eigenvector extracted from original MRIs which is a patch of 3 × 3 around the voxel. The kernel dictionary learning is used to extract the non-linear features to construct two adaptive dictionaries for healthy and pathologically tissues respectively. A kernel-clustering algorithm based on dictionary learning is developed to code the voxels, then the linear discrimination method is used to classify the target pixels. In the end, the flood-fill operation is used to improve the segmentation quality. The results demonstrate that the method based on kernel sparse coding has better capacity and higher segmentation accuracy with low computation cost. [ABSTRACT FROM AUTHOR]

• We develop a single-stage knowledge distillation framework that successively trains the teacher and student networks in the same training iteration, which can be used for brain tumor segmentation on limited MR modality. • As a practical verification of the feasibility of the proposed segmentation approach to be generalized to clinical usage, we further test our model using brain dataset collected from the local hospital. • Experimental results of proposed model are better than other benchmark methods on both the BraTS dataset and our local dataset. Precisely segmenting brain tumors using multimodal Magnetic Resonance Imaging (MRI) is an essential task for early diagnosis, disease monitoring, and surgical planning. Unfortunately, the complete four image modalities utilized in the well-known BraTS benchmark dataset: T1, T2, Fluid-Attenuated Inversion Recovery (FLAIR), and T1 Contrast-Enhanced (T1CE) are not regularly acquired in clinical practice due to the high cost and long acquisition time. Rather, it is common to utilize limited image modalities for brain tumor segmentation. In this paper, we propose a single stage learning of knowledge distillation algorithm that derives information from the missing modalities for better segmentation of brain tumors. Unlike the previous works that adopted a two-stage framework to distill the knowledge from a pre-trained network into a student network, where the latter network is trained on limited image modality, we train both models simultaneously using a single-stage knowledge distillation algorithm. We transfer the information by reducing the redundancy from a teacher network trained on full image modalities to the student network using Barlow Twins loss on a latent-space level. To distill the knowledge on the pixel level, we further employ a deep supervision idea that trains the backbone networks of both teacher and student paths using Cross-Entropy loss. We demonstrate that the proposed single-stage knowledge distillation approach enables improving the performance of the student network in each tumor category with overall dice scores of 91.11% for Tumor Core, 89.70% for Enhancing Tumor, and 92.20% for Whole Tumor in the case of only using the FLAIR and T1CE images, outperforming the state-of-the-art segmentation methods. The outcomes of this work prove the feasibility of exploiting the knowledge distillation in segmenting brain tumors using limited image modalities and hence make it closer to clinical practices. [Display omitted] [ABSTRACT FROM AUTHOR]