Search

Data privacy mechanisms are essential for rapidly scaling medical training databases to capture the heterogeneity of patient data distributions toward robust and generalizable machine learning systems. In the current COVID-19 pandemic, a major focus of artificial intelligence (AI) is interpreting chest CT, which can be readily used in the assessment and management of the disease. This paper demonstrates the feasibility of a federated learning method for detecting COVID-19 related CT abnormalities with external validation on patients from a multinational study. We recruited 132 patients from seven multinational different centers, with three internal hospitals from Hong Kong for training and testing, and four external, independent datasets from Mainland China and Germany, for validating model generalizability. We also conducted case studies on longitudinal scans for automated estimation of lesion burden for hospitalized COVID-19 patients. We explore the federated learning algorithms to develop a privacy-preserving AI model for COVID-19 medical image diagnosis with good generalization capability on unseen multinational datasets. Federated learning could provide an effective mechanism during pandemics to rapidly develop clinically useful AI across institutions and countries overcoming the burden of central aggregation of large amounts of sensitive data.

This paper presents a novel mixed reality based navigation system for accurate respiratory liver tumor punctures in radiofrequency ablation (RFA). Our system contains an optical see-through head-mounted display device (OST-HMD), Microsoft HoloLens for perfectly overlaying the virtual information on the patient, and a optical tracking system NDI Polaris for calibrating the surgical utilities in the surgical scene. Compared with traditional navigation method with CT, our system aligns the virtual guidance information and real patient and real-timely updates the view of virtual guidance via a position tracking system. In addition, to alleviate the difficulty during needle placement induced by respiratory motion, we reconstruct the patient-specific respiratory liver motion through statistical motion model to assist doctors precisely puncture liver tumors. The proposed system has been experimentally validated on vivo pigs with an accurate real-time registration approximately 5-mm mean FRE and TRE, which has the potential to be applied in clinical RFA guidance.

Microelectrode recordings (MERs) are a significant clinical indicator for sweet spots identification of implanted electrodes during deep brain stimulation of the subthalamic nucleus (STN) surgery. As 1D MERs signals have the unboundedness, large-range, large-amount and time-dependent characteristics, the purpose of this study is to propose an automatic and precise identification method of sweet spots from MERs, reducing the time-consuming and labor-intensive human annotations. We propose an automatic identification method of sweet spots from MERs for electrodes implantation in STN-DBS. To better imitate the surgeons’ observation and obtain more intuitive contextual information, we first employ the 2D Gramian angular summation field (GASF) images generated from MERs data to perform the sweet spots determination for electrodes implantation. Then, we introduce the convolutional block attention module into convolutional neural network (CNN) to identify the 2D GASF images of sweet spots for electrodes implantation. Experimental results illustrate that the identification result of our method is consistent with the result of doctor’s decision, while our method can achieve the accuracy and precision of 96.72% and 98.97%, respectively, which outperforms state-of-the-art for intraoperative sweet spots determination. The proposed method is the first time to automatically and accurately identify sweet spots from MERs for electrodes implantation by the combination an advanced time series-to-image encoding way with CBAM-enhanced networks model. Our method can assist neurosurgeons in automatically detecting the most likely locations of sweet spots for electrodes implantation, which can provide an important indicator for target selection while it reduces the localization error of the target during STN-DBS surgery.

The terminal complement complex C5b-9 plays an important role in acute ischemic stroke (AIS) and carotid atherosclerosis. However, the associations between serum C5b-9, the severity and outcome of AIS, and the stability of carotid plaques have not been well investigated. In this clinical study, 70 patients with AIS and 70 healthy controls were enrolled. Serum C5b-9 levels at 72 h after stroke onset were measured by enzyme-linked immunosorbent assay (ELISA). Infarct size, the National Institutes of Health Stroke Scale (NIHSS), the 90-day modified Rankin Scale (mRS), and carotid plaque and stenosis were evaluated. Serum C5b-9 levels were significantly higher in AIS patients than in healthy controls (p < 0.001) and were correlated with infarction sizes (p = 0.045) and the NIHSS (P = 0.035). Furthermore, 90-day mRS analysis demonstrated that the patients with poor outcomes had higher serum C5b-9 levels than those with good outcomes (P < 0.001). Moreover, serum C5b-9 levels in AIS patients with unstable carotid plaques were much higher than in those with stable carotid plaques (P = 0.009). Multivariate logistic regression indicated that C5b-9 could be an independent risk factor for AIS (P < 0.001) and unstable carotid plaques (P = 0.015). Therefore, complement complex C5b-9 may be a potential biomarker in predicting the severity and outcome, as well as the stability of carotid plaques, in AIS patients.

CT-guided radiofrequency ablation (RFA) has evolved rapidly over the past decade and become a widely accepted treatment option for patients with liver tumors. However, it is hard for doctors to locate tumors precisely while avoid damaging the surrounding risk structures with 2D CT images, which only provides limited static information, especially in case of respiratory motion. This paper presents a novel augmented reality guidance modality for improving the precision of liver tumors punctures by providing visual cue of 3D personalized anatomy with respiratory motion. Optical see-through display devices Epson MoveRio BT300 and Microsoft HoloLens are used to mix pre-operative 3D personalized data and intra-operative physical scene. Here an augmented reality based surgical navigation pipeline is proposed to achieve the transformation from raw medical data to virtual guidance information and precisely superimpose this information onto real experimental animal. In addition, to alleviate the difficulty during needle placement induced by respiratory motion, we proposed a correlation model to real-timely predict the tumor position via regression based respiration state estimation and the statistical tumor motion model. We experimentally validated the proposed system on in vivo beagle dogs with artificial lesion, which can effectively improve the puncture efficiency and precision. The proposed augmented reality modality is a general strategy to guide the doctors perform precise percutaneous puncture under respiration conditions and has the potential to be used for other surgical navigation tasks.

This paper presents a novel mixed reality (MR) guidance method for liver tumors radiofrequency ablation (RFA). Compared with traditional computed tomography (CT)-guided method, our system can provide a more natural and intuitive surgical mode for surgeons. In essence, our system is a holographic navigation platform, which projects a MR overlay onto the patient via HoloLens during RFA. We first reconstruct the patient-specific anatomy structure from the CT images of abdominal phantom. Then, a tailored precise registration method is employed to map the virtual-real spatial information. In addition, considering that tumor shifting during biopsy severely impacts the accuracy of RFA, our guidance system involves a motion compensation computation through data-driven physically-based modeling in holographic environment. In experiments, we conduct a user study on the comparison trial between MR-guided and CT-guided biopsy. User feedback demonstrates that our MR guidance method for needle placement procedure has the potential to simplify the operation, reduce the operation difficulty, shorten the operation time, and raise the operation precision.

Highlights • This work presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MM-WHS) challenge, in conjunction with MICCAI 2017. • This work introduces the related information to the challenge, discusses the results from the conventional methods and deep learning-based algorithms, and provides insights to the future research. • The challenge provides a fair and intuitive comparison framework for methods developed and being developed for WHS. • The challenge provides the training datasets with manually delineated ground truths and evaluation for an ongoing development of MM-WHS algorithms., Graphical abstract This manuscript presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MMWHS) challenge, in conjunction with MICCAI-STACOM 2017. The challenge provides 120 three-dimensional cardiac images covering the whole heart, including 60 CT and 60 MRI volumes, all acquired in clinical environments with manual delineation. Ten algorithms for CT data and eleven algorithms for MRI data, submitted from twelve groups, have been evaluated. The results show that many of the deep learning (DL) based methods achieved high accuracy, even though the number of training datasets were limited. Several of them also reported poor results in the blinded evaluation, probably due to over fitting in their training. The conventional algorithms, mainly based on multi-atlas segmentation, demonstrated robust and stable performance, even though the accuracy is not as good as the best DL method in CT segmentation. The challenge, including provision of the annotated training data and the blinded evaluation for submitted algorithms on the test data, continues as an ongoing benchmarking resource., Knowledge of whole heart anatomy is a prerequisite for many clinical applications. Whole heart segmentation (WHS), which delineates substructures of the heart, can be very valuable for modeling and analysis of the anatomy and functions of the heart. However, automating this segmentation can be challenging due to the large variation of the heart shape, and different image qualities of the clinical data. To achieve this goal, an initial set of training data is generally needed for constructing priors or for training. Furthermore, it is difficult to perform comparisons between different methods, largely due to differences in the datasets and evaluation metrics used. This manuscript presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MM-WHS) challenge, in conjunction with MICCAI 2017. The challenge provided 120 three-dimensional cardiac images covering the whole heart, including 60 CT and 60 MRI volumes, all acquired in clinical environments with manual delineation. Ten algorithms for CT data and eleven algorithms for MRI data, submitted from twelve groups, have been evaluated. The results showed that the performance of CT WHS was generally better than that of MRI WHS. The segmentation of the substructures for different categories of patients could present different levels of challenge due to the difference in imaging and variations of heart shapes. The deep learning (DL)-based methods demonstrated great potential, though several of them reported poor results in the blinded evaluation. Their performance could vary greatly across different network structures and training strategies. The conventional algorithms, mainly based on multi-atlas segmentation, demonstrated good performance, though the accuracy and computational efficiency could be limited. The challenge, including provision of the annotated training data and the blinded evaluation for submitted algorithms on the test data, continues as an ongoing benchmarking resource via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/mmwhs/).

This paper presents a novel augmented reality (AR)-based neurosurgical training simulator which provides a very natural way for surgeons to learn neurosurgical skills. Surgical simulation with bimanual haptic interaction is integrated in this work to provide a simulated environment for users to achieve holographic guidance for pre-operative training. To achieve the AR guidance, the simulator should precisely overlay the 3D anatomical information of the hidden target organs in the patients in real surgery. In this regard, the patient-specific anatomy structures are reconstructed from segmented brain magnetic resonance imaging. We propose a registration method for precise mapping of the virtual and real information. In addition, the simulator provides bimanual haptic interaction in a holographic environment to mimic real brain tumor resection. In this study, we conduct AR-based guidance validation and a user study on the developed simulator, which demonstrate the high accuracy of our AR-based neurosurgery simulator, as well as the AR guidance mode’s potential to improve neurosurgery by simplifying the operation, reducing the difficulty of the operation, shortening the operation time, and increasing the precision of the operation.

Cardiac magnetic resonance (CMR) segmentation has been a glaring medical imaging issue for the past decades. The existing convolutional networks used for CMR segmentation lack adequate communication between semantic and spatial information. In this paper, we propose an aggregated parallel cross-scale pyramid network (APCP-Net) for CMR segmentation, which adopts parallel cross-scale spatial pyramid to enhance the ability of multi-scale feature fusion. To alleviate the computational complexity of the network without losing performance and increasing time consumption, we present a convolution decomposition strategy which decomposes a regular convolutional layer into several lightweight ones. Furthermore, deep aggregation is used among parallel spatial pyramids to promote the representation ability of our APCP-Net. We evaluate our APCP-Net on the segmentation dataset of ACDC 2017 Challenge. Experimental results demonstrate that our APCP-Net is comparable to the state-of-the-art methods in the case where the parameter amount is 54 times less than the existing methods.

Background Segmentation of anatomical structures of the heart from cardiac magnetic resonance images (MRI) has a significant impact on the quantitative analysis of the cardiac contractile function. Although deep convolutional neural networks (ConvNets) have achieved considerable success in medical imaging segmentation, it is still a challenging task for existing deep ConvNets to precisely and automatically segment multiple heart structures from cardiac MRI. This paper presents a novel recurrent interleaved attention network (RIANet) to comprehensively tackle this issue. Method The proposed RIANet can efficiently reuse parameters to encode richer representative features via introducing a recurrent feedback structure, Clique Block, which incorporates both forward and backward connections between different layers with the same resolution. Further, we integrate a plug-and-play interleaved attention (IA) block to modulate the information passed to the decoding stage of RIANet by effectively fusing multi-level contextual information. In addition, we improve the discrimination capability of our RIANet through a deep supervision mechanism with weighted losses. Results The performance of RIANet has been extensively validated in the segmentation contest of the ACDC 2017 challenge held in conjunction with MICCAI 2017, with mean Dice scores of 0.942 (left ventricular), 0.923 (right ventricular) and 0.910 (myocardium) for cardiac MRI segmentation. Besides, we visualize intermediate features of our RIANet using guided backpropagation, which can intuitively depict the effects of our proposed components in feature representation. Conclusion Experimental results demonstrate that our RIANet have achieved competitive segmentation results with fewer parameters compared with the state-of-the-art approaches, corroborating the effectiveness and robustness of our proposed RIANet.

Segmentation of medical images, particularly late gadolinium-enhanced magnetic resonance imaging (LGE-MRI) used for visualizing diseased atrial structures, is a crucial first step for ablation treatment of atrial fibrillation. However, direct segmentation of LGE-MRIs is challenging due to the varying intensities caused by contrast agents. Since most clinical studies have relied on manual, labor-intensive approaches, automatic methods are of high interest, particularly optimized machine learning approaches. To address this, we organized the 2018 Left Atrium Segmentation Challenge using 154 3D LGE-MRIs, currently the world's largest atrial LGE-MRI dataset, and associated labels of the left atrium segmented by three medical experts, ultimately attracting the participation of 27 international teams. In this paper, extensive analysis of the submitted algorithms using technical and biological metrics was performed by undergoing subgroup analysis and conducting hyper-parameter analysis, offering an overall picture of the major design choices of convolutional neural networks (CNNs) and practical considerations for achieving state-of-the-art left atrium segmentation. Results show that the top method achieved a Dice score of 93.2% and a mean surface to surface distance of 0.7 mm, significantly outperforming prior state-of-the-art. Particularly, our analysis demonstrated that double sequentially used CNNs, in which a first CNN is used for automatic region-of-interest localization and a subsequent CNN is used for refined regional segmentation, achieved superior results than traditional methods and machine learning approaches containing single CNNs. This large-scale benchmarking study makes a significant step towards much-improved segmentation methods for atrial LGE-MRIs, and will serve as an important benchmark for evaluating and comparing the future works in the field. Furthermore, the findings from this study can potentially be extended to other imaging datasets and modalities, having an impact on the wider medical imaging community. (C) 2020 Elsevier B.V. All rights reserved.

Mitral valve repair is a complex operation, in which the functionality of incompetent mitral valve is reconstructed by surgical techniques. Simulation-based surgical planning system, allowing surgeons to simulate and compare potential repair strategies, could greatly improve surgical outcomes. This paper presents a practical computational framework, combining the Total Lagrangian Explicit Dynamics Finite Element Method (TLED FEM) and Smoothed Particle Hydrodynamics (SPH), to solve the interaction problem of blood and immersed mitral valves. With this completed pipeline, we can not only predict the mechanical behavior of mitral valve, but also analyze the transvalvular pressures distributed on valve leaflets. The experimental results demonstrate that our method has the potential to be applied in surgical planning simulator of mitral valve repair.

This paper presents a novel augmented reality (AR) interactive environment for neurosurgical training. Comparing with traditional virtual reality based neurosurgical simulator, our system provides a more natural and intuitive fashion for surgeons. To achieve holographic visualization of virtual brain on 3D-printed skull (workspace), the first step is to reconstruct the personalized anatomy structure from segmented MR imaging. Then, tailored to the computational power of HoloLens, we employ the mass-spring method to model the mechanical response of brain. After that, a precise registration method is employed to map the virtual-real spatial information, which can overlay the virtual operative brain on workspace. In addition, bimanual haptic interface is also integrated into our simulator, which is more similar with real neurosurgery. In experiments, we conduct accuracy validation on our registration method, as well as the validity test on the developed simulators. The results demonstrate that our simulator can provide high-accuracy augmented visualization effects and deep immersion for novice surgeons.

PurposeIn ultrasound-guided High Intensity Focused Ultrasound (HIFU) therapy, the target tissue (such as a tumor) often moves and/or deforms in response to an external force. This problem creates difficulties in treating patients and can lead to the destruction of normal tissue. In order to solve this problem, we present a novel method to model and predict the movement and deformation of the target tissue during ultrasound-guided HIFU therapy.MethodsOur method computationally predicts the position of the target tissue under external force. This prediction allows appropriate adjustments in the focal region during the application of HIFU so that the treatment head is kept aligned with the diseased tissue through the course of therapy. To accomplish this goal, we utilize the cow tissue as the experimental target tissue to collect spatial sequences of ultrasound images using the HIFU equipment. A Geodesic Localized Chan-Vese (GLCV) model is developed to segment the target tissue images. A 3D target tissue model is built based on the segmented results. A versatile particle framework is constructed based on Smoothed Particle Hydrodynamics (SPH) to model the movement and deformation of the target tissue. Further, an iterative parameter estimation algorithm is utilized to determine the essential parameters of the versatile particle framework. Finally, the versatile particle framework with the determined parameters is used to estimate the movement and deformation of the target tissue.ResultsTo validate our method, we compare the predicted contours with the ground truth contours. We found that the lowest, highest and average Dice Similarity Coefficient (DSC) values between predicted and ground truth contours were, respectively, 0.9615, 0.9770 and 0.9697.ConclusionOur experimental result indicates that the proposed method can effectively predict the dynamic contours of the moving and deforming tissue during ultrasound-guided HIFU therapy.

Background Biomechanical deformable volumetric registration can help improve safety of surgical interventions by ensuring the operations are extremely precise. However, this technique has been limited by the accuracy and the computational efficiency of patient-specific modeling. Methods This study presents a tissue–tissue coupling strategy based on penalty method to model the heterogeneous behavior of deformable body, and estimate the personalized tissue–tissue coupling parameters in a data-driven way. Moreover, considering that the computational efficiency of biomechanical model is highly dependent on the mechanical resolution, a practical coarse-to-fine scheme is proposed to increase runtime efficiency. Particularly, a detail enrichment database is established in an offline fashion to represent the mapping relationship between the deformation results of high-resolution hexahedral mesh extracted from the raw medical data and a newly constructed low-resolution hexahedral mesh. At runtime, the mechanical behavior of human organ under interactions is simulated with this low-resolution hexahedral mesh, then the microstructures are synthesized in virtue of the detail enrichment database. Results The proposed method is validated by volumetric registration in an abdominal phantom compression experiments. Our personalized heterogeneous deformable model can well describe the coupling effects between different tissues of the phantom. Compared with high-resolution heterogeneous deformable model, the low-resolution deformable model with our detail enrichment database can achieve 9.4× faster, and the average target registration error is 3.42 mm, which demonstrates that the proposed method shows better volumetric registration performance than state-of-the-art. Conclusions Our framework can well balance the precision and efficiency, and has great potential to be adopted in the practical augmented reality image-guided robotic systems.