Search

Medical image segmentation and registration are two tasks to analyze the anatomical structures in clinical research. Still, deep-learning solutions utilizing the connections between segmentation and registration remain underdiscovered. This article designs a joint learning framework named RSegNet that can realize concurrent deformable registration and segmentation by minimizing an integrated loss function, including three parts: diffeomorphic registration loss, segmentation similarity loss, and dual-consistency supervision loss. The probabilistic diffeomorphic registration branch could benefit from the auxiliary segmentations available from the segmentation branch to achieve anatomical consistency and better deformation regularity by dual-consistency supervision. Simultaneously, the segmentation performance could also be improved by data augmentation based on the registration with well-behaved diffeomorphic guarantees. Experiments on the human brain 3-D magnetic resonance images have been implemented to demonstrate the effectiveness of our approach. We trained and validated RSegNet with 1000 images and tested its performances on four public datasets, which shows that our method successfully yields concurrent improvements of both segmentation and registration compared with separately trained networks. Specifically, our method can increase the accuracy of segmentation and registration by 7.0% and 1.4%, respectively, in terms of Dice scores. Note to Practitioners—Registration and segmentation of medical images are two significant tasks in medical research and clinical application. However, most existing approaches consider these two tasks independently while neglecting the potential association between them. Therefore, we suggest a new approach that combines these two tasks into one joint deep learning framework, boosting registration, and segmentation performance by introducing dual-consistency supervision. Besides, our framework could generate outputs within 1 s by taking an affinely aligned medical image pair as input, which is suitable for time-critical requirements in a clinic. We tested it on four public datasets and achieved state-of-the-art performance to demonstrate the proposed method’s feasibility and robustness. Furthermore, our proposed RSegNet is a general learning framework suitable for various image modalities and anatomical structures. Hence, we expect our framework to serve as a practical clinical tool to speed up medical image analysis procedures and improve diagnostic accuracy. [ABSTRACT FROM AUTHOR]

Medical image segmentation and registration have vital roles in computer-assisted diagnosis procedures, challenging tasks suffering from various limitations and artifacts inside images. Recently, deep learning techniques accomplish these two tasks and achieve outstanding performances. However, most deep learning-based methods overlook the potential correlation between each other. In this paper, an unsupervised probabilistic model named U-RSNet is proposed to realize concurrent medical image registration and segmentation in one framework. Specifically, the unsupervised segmentation branch is derived from Bayesian inference. The prior warped atlas for segmentation can be obtained by deforming a known probabilistic atlas by the corresponding invertible deformation field with a well-behaved diffeomorphic guarantee, which can perfectly integrate these two tasks to form a complete intelligent prediction system. In this case, the segmentation performance could be largely improved based on the warped probabilistic atlas obtained from the registration branch. Experiments on human brain 3D magnetic resonance images have demonstrated the effectiveness of our approach. We trained and validated U-RSNet with 1000 images and tested its performances on four public datasets. We showed our method successfully realized concurrent segmentation and registration and yielded better segmentation results than a separately trained network. [ABSTRACT FROM AUTHOR]

Magnetic resonance imaging (MRI) is beneficial for imaging-guided procedures because it provides higher resolution images and better soft tissue contrast than computed tomography (CT), ultrasound, and X-ray. MRI can be used to streamline diagnostics and treatment because it does not require patients to be repositioned between scans of different areas of the body. It is even possible to use MRI to visualize, power, and control medical devices inside the human body to access remote locations and perform minimally invasive procedures. Therefore, MR conditional medical devices have the potential to improve a wide variety of medical procedures; this potential is explored in terms of practical considerations pertaining to clinical applications and the MRI environment. Recent advancements in this field are introduced with a review of clinically relevant research in the areas of interventional tools, endovascular microbots, and closed-loop controlled MRI robots. Challenges related to technology and clinical feasibility are discussed, including MRI based propulsion and control, navigation of medical devices through the human body, clinical adoptability, and regulatory issues. The development of MRI-powered medical devices is an emerging field, but the potential clinical impact of these devices is promising. [ABSTRACT FROM AUTHOR]

[Display omitted] • Propose an FCN architecture using octave partial convolution and skip connection with scSE (skip-scSE). • Design a novel decoder efficiently using a skip-scSE module. • Design a multi-modal MRI to single MRI modality synthesizing model using proposed FCN as a generator and encoder–decoder discriminator using the cGAN learning approach. • Generate missing modalities for the TCGA dataset using the available modality (or modalities) and predict segmentation using both available and synthesized modalities. • Design a radiogenomic model by combining radiomic features such as intensity, shape, and geometry with gene expression profiling data. The accurate prognosis of glioblastoma multiforme (GBM) plays an essential role in planning correlated surgeries and treatments. The conventional models of survival prediction rely on radiomic features using magnetic resonance imaging (MRI). In this paper, we propose a radiogenomic overall survival (OS) prediction approach by incorporating gene expression data with radiomic features such as shape, geometry, and clinical information. We exploit TCGA (The Cancer Genomic Atlas) dataset and synthesize the missing MRI modalities using a fully convolutional network (FCN) in a conditional generative adversarial network (cGAN). Meanwhile, the same FCN architecture enables the tumor segmentation from the available and the synthesized MRI modalities. The proposed FCN architecture comprises octave convolution (OctConv) and a novel decoder, with skip connections in spatial and channel squeeze & excitation (skip-scSE) block. The OctConv can process low and high-frequency features individually and improve model efficiency by reducing channel-wise redundancy. Skip-scSE applies spatial and channel-wise excitation to signify the essential features and reduces the sparsity in deeper layers learning parameters using skip connections. The proposed approaches are evaluated by comparative experiments with state-of-the-art models in synthesis, segmentation, and overall survival (OS) prediction. We observe that adding missing MRI modality improves the segmentation prediction, and expression levels of gene markers have a high contribution in the GBM prognosis prediction, and fused radiogenomic features boost the OS estimation. [ABSTRACT FROM AUTHOR]

Glioblastoma multiforme (GBM) are malignant brain tumors, associated with poor overall survival (OS). This study aims to predict OS of GBM patients (in days) using a regression framework and assess the impact of tumor shape features on OS prediction. Multi-channel MR image derived texture features, tumor shape, and volumetric features, and patient age were obtained for 163 GBM patients. In order to assess the impact of tumor shape features on OS prediction, two feature sets, with and without tumor shape features, were created. For the feature set with tumor shape features, the mean prediction error (MPE) was 14.6 days and its 95% confidence interval (CI) was 195.8 days. For the feature set excluding shape features, the MPE was 17.1 days and its 95% CI was observed to be 212.7 days. The coefficient of determination (R2) value obtained for the feature set with shape features was 0.92, while it was 0.90 for the feature set excluding shape features. Although marginal, inclusion of shape features improves OS prediction in GBM patients. The proposed OS prediction method using regression provides good accuracy and overcomes the limitations of GBM OS classification, like choosing data-derived or pre-decided thresholds to define the OS groups. Graphical abstract Two feature sets: with and without tumor shape features were extracted from T1-weighted contrast-enhanced, T2-weighted and FLAIR MRI. These feature sets were analyzed using the Mean Prediction Error (MPE) and its 95% Confidence Interval (CI) obtained from the Bland-Altman plot, along with the coefficient of determination (R2) value to assess the impact of tumor shape features on overall survival prediction of glioblastoma multiforme patients. [ABSTRACT FROM AUTHOR]

Glioblastoma multiforme (GBM) are aggressive brain tumors, which lead to poor overall survival (OS) of patients. OS prediction of GBM patients provides useful information for surgical and treatment planning. Radiomics research attempts at predicting disease prognosis, thus providing beneficial information for personalized treatment from a variety of imaging features extracted from multiple MR images. In this study, MR image derived texture features, tumor shape and volumetric features, and patient age were obtained for 163 patients. OS group prediction was performed for both 2-class (short and long) and 3-class (short, medium and long) survival groups. Support vector machine classification based recursive feature elimination method was used to perform feature selection. The performance of the classification model was assessed using 5-fold cross-validation. The 2-class and 3-class OS group prediction accuracy obtained were 98.7% and 88.95% respectively. The shape features used in this work have been evaluated for OS prediction of GBM patients for the first time. The feature selection and prediction scheme implemented in this study yielded high accuracy for both 2-class and 3-class OS group predictions. This study was performed using routinely acquired MR images for GBM patients, thus making the translation of this work into a clinical setup convenient. [ABSTRACT FROM AUTHOR]

Magnetic tracking technology is emerging to provide an occlusion-free tracking scheme for the estimation of full pose (position and orientation) of various instruments. This brings substantial benefits for intracorporeal applications, such as for tracking of flexible or wireless endoscopic devices, and thus is significant for further computer-assisted diagnosis, interventions, and surgeries. Toward efficient magnetic tracking, a 6-D magnetic localization and orientation method is proposed in this paper. An annular permanent magnet is mounted on the exterior surface of a capsule. With a magnetic sensor array, the magnetic field can be measured and the capsule’s 3-D location and 3-D orientation information can be estimated based on proposed closed-form analytical model of annular magnet and particle swarm optimization algorithm. Magnetic dipole model and Levenberg–Marquardt algorithm are used to improve the speed and accuracy of estimation. Extensive simulation experiments show that the localization and orientation method works well with good position and orientation accuracy. [ABSTRACT FROM AUTHOR]