945 results on '"DingGang Shen"'
Search Results
2. Hierarchical Organ-Aware Total-Body Standard-Dose PET Reconstruction From Low-Dose PET and CT Images
- Author
-
Jiadong Zhang, Zhiming Cui, Caiwen Jiang, Shanshan Guo, Fei Gao, and Dinggang Shen
- Subjects
Artificial Intelligence ,Computer Networks and Communications ,Software ,Computer Science Applications - Published
- 2023
3. AutoEncoder-Driven Multimodal Collaborative Learning for Medical Image Synthesis
- Author
-
Bing Cao, Zhiwei Bi, Qinghua Hu, Han Zhang, Nannan Wang, Xinbo Gao, and Dinggang Shen
- Subjects
Artificial Intelligence ,Computer Vision and Pattern Recognition ,Software - Published
- 2023
4. Constructing Multi-View High-Order Functional Connectivity Networks for Diagnosis of Autism Spectrum Disorder
- Author
-
Ning Mao, Feng Zhao, Kim-Han Thung, Seong-Whan Lee, Xiangfei Zhang, and Dinggang Shen
- Subjects
Brain Mapping ,Time Factors ,medicine.diagnostic_test ,Series (mathematics) ,Autism Spectrum Disorder ,Computer science ,business.industry ,Functional connectivity ,Biomedical Engineering ,Brain ,Pattern recognition ,medicine.disease ,Magnetic Resonance Imaging ,Correlation ,Discriminative model ,Autism spectrum disorder ,medicine ,Humans ,Central moment ,Artificial intelligence ,High order ,Functional magnetic resonance imaging ,business - Abstract
Brain functional connectivity network (FCN) based on resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used to identify neuropsychiatric disorders such as autism spectrum disorder (ASD). Most existing FCN-based methods only estimate the correlation between brain regions of in terest (ROIs), without exploring more informative higher-level inter actions among multiple ROIs which could be beneficial to disease diagnosis. To fully explore the discriminative information provided by different brain networks, a cluster-based multi-view high-order FCN (Ho-FCN) framework is proposed in this paper. Specifically, we first group the functional connectivity (FC) time series into different clusters and compute the multi-order central moment series for the FC time series in each cluster. Then we utilize the correlation of central moment series between different clusters to reveal the high-order FC relationships among multiple ROIs. In addition, to address the phase mismatch issue in conventional FCNs, we also adopt the central moments of the correlation time series as the temporal-invariance features to capture the dynamic characteristics of low-order dynamic FCN (Lo-D-FCN). Experimental results on the ABIDE dataset validate that: 1) the proposed multi-view Ho-FCNs is able to explore rich discriminative information for ASD diagnosis; 2) the phase mismatch issue can be well circumvented by using central moments; and 3) the combination of different types of FCNs can significantly improve the diagnostic accuracy of ASD (86.2%).
- Published
- 2022
5. Learning Brain Functional Networks With Latent Temporal Dependency for MCI Identification
- Author
-
Dinggang Shen, Yining Zhang, Seong-Whan Lee, Limei Zhang, Lishan Qiao, and Yanfang Xue
- Subjects
Sequence ,Dependency (UML) ,medicine.diagnostic_test ,business.industry ,Computer science ,Biomedical Engineering ,Brain ,Pattern recognition ,Latent variable ,medicine.disease ,Magnetic Resonance Imaging ,Functional networks ,Identification (information) ,Encoding (memory) ,Image Interpretation, Computer-Assisted ,medicine ,Humans ,Cognitive Dysfunction ,Artificial intelligence ,Mild cognitive impairment (MCI) ,business ,Functional magnetic resonance imaging ,Algorithms - Abstract
Resting-state functional magnetic resonance imaging (rs-fMRI) has become a popular non-invasive way of diagnosing neurological disorders or their early stages by probing functional connectivity between different brain regions of interest (ROIs) across subjects. In the past decades, researchers have proposed many methods to estimate brain functional networks (BFNs) based on blood-oxygen-level-dependent (BOLD) signals captured by rs-fMRI. However, most of the existing methods estimate BFNs under the assumption that signals are independently sampled, which ignores the temporal dependency and sequential order of different time points (or volumes). To address this problem, in this paper, we first propose a novel BFN estimation model by introducing a latent variable to control the sequence of volumes for encoding the temporal dependency and sequential information of signals into the estimated BFNs. Then, we develop an efficient learning algorithm to solve the proposed model by the alternating optimization scheme. To verify the effectiveness of the proposed method, the estimated BFNs are used to identify subjects with mild cognitive impairment (MCIs) from normal controls (NCs). Experimental results show that our method outperforms the baseline methods in the sense of classification performance.
- Published
- 2022
6. Cross-level Feature Aggregation Network for Polyp Segmentation
- Author
-
Tao Zhou, Yi Zhou, Kelei He, Chen Gong, Jian Yang, Huazhu Fu, and Dinggang Shen
- Subjects
Artificial Intelligence ,Signal Processing ,Computer Vision and Pattern Recognition ,Software - Published
- 2023
7. A review of deep learning-based three-dimensional medical image registration methods
- Author
-
Dinggang Shen, Ruijie Yang, Xinzhi Teng, Chenyang Liu, Jing Cai, Tian Li, Ge Ren, and Haonan Xiao
- Subjects
Computer science ,business.industry ,Deep learning ,Image registration ,Radiology, Nuclear Medicine and imaging ,Computer vision ,Review Article ,Artificial intelligence ,business - Abstract
Medical image registration is a vital component of many medical procedures, such as image-guided radiotherapy (IGRT), as it allows for more accurate dose-delivery and better management of side effects. Recently, the successful implementation of deep learning (DL) in various fields has prompted many research groups to apply DL to three-dimensional (3D) medical image registration. Several of these efforts have led to promising results. This review summarized the progress made in DL-based 3D image registration over the past 5 years and identify existing challenges and potential avenues for further research. The collected studies were statistically analyzed based on the region of interest (ROI), image modality, supervision method, and registration evaluation metrics. The studies were classified into three categories: deep iterative registration, supervised registration, and unsupervised registration. The studies are thoroughly reviewed and their unique contributions are highlighted. A summary is presented following a review of each category of study, discussing its advantages, challenges, and trends. Finally, the common challenges for all categories are discussed, and potential future research topics are identified.
- Published
- 2021
8. Liver DCE‐MRI registration based on sparse recovery of contrast agent curves
- Author
-
Meiyan Huang, Yuhang Sun, Yujia Zhou, Qiaoyun Zhu, Qianjin Feng, and Dinggang Shen
- Subjects
Series (mathematics) ,Computer science ,business.industry ,media_common.quotation_subject ,Liver Neoplasms ,Subtraction ,Contrast Media ,Image registration ,Pattern recognition ,General Medicine ,Sparse approximation ,Residual ,Magnetic Resonance Imaging ,Standard deviation ,Dynamic contrast-enhanced MRI ,Humans ,Contrast (vision) ,Artificial intelligence ,business ,Algorithms ,media_common - Abstract
Purpose Dynamic contrast-enhanced MRI (DCE-MRI) registration is a challenging task because of the effect of remarkable intensity changes caused by contrast agent injections. Unrealistic deformation usually occurs by using traditional intensity-based algorithms. To alleviate the effect of contrast agent on registration, we proposed a DCE-MRI registration strategy and investigated the registration performance on the clinical DCE-MRI time series of liver. Method We reconstructed the time-intensity curves of the contrast agent through sparse representation with a predefined dictionary whose columns were the time-intensity curves with high correlations with respect to a preselected contrast agent curve. After reshaping 1D-reconstructed contrast agent time-intensity curves into a 4D contrast agent time series, we aligned the original time series to the reconstructed contrast agent time series through traditional free-form deformation (FFD) registration scheme combined with a residual complexity (RC) similarity and an iterative registration strategy. This study included the DCE-MRI time series of 20 patients with liver cancer. Results Qualitatively, the time-cut images and subtraction images of different registration methods did not obviously differ. Quantitatively, the mean (standard deviation) of temporal intensity smoothness of all the patients achieved 54.910 (18.819), 54.609 (18.859), and 53.391 (19.031) in FFD RC, RDDR, Zhou et al.'s method and the proposed method, respectively. The mean (standard deviation) of changes in the lesion volume were 0.985 (0.041), 0.983 (0.041), 0.981 (0.046), and 0.989 (0.036) in FFD RC, RDDR, Zhou et al.'s method and the proposed method. Conclusion Our proposed method would be an effective registration strategy for DCE-MRI time series, and its performance was comparable with that of three advanced registration methods.
- Published
- 2021
9. Edge-preserving MRI image synthesis via adversarial network with iterative multi-scale fusion
- Author
-
Bo Zhan, Yanmei Luo, Jiliu Zhou, Li Zhi'ang, Yan Wang, Dinggang Shen, Dong Nie, and Xi Wu
- Subjects
0209 industrial biotechnology ,Modality (human–computer interaction) ,business.industry ,Computer science ,Cognitive Neuroscience ,Pattern recognition ,02 engineering and technology ,Computer Science Applications ,Task (project management) ,Image (mathematics) ,020901 industrial engineering & automation ,Artificial Intelligence ,Feature (computer vision) ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Enhanced Data Rates for GSM Evolution ,Artificial intelligence ,business ,Representation (mathematics) ,Encoder ,Generator (mathematics) - Abstract
Magnetic resonance imaging (MRI) is a major imaging technique for studying neuroanatomy. By applying different pulse sequences and parameters, different modalities can be generated regarding the same anatomical structure, which can provide complementary information for diagnosis. However, limited by the scanning time and related cost, multiple different modalities are often not available for the same patient in clinic. Recently, many methods have been proposed for cross-modality MRI synthesis, but most of them only consider pixel-level differences between the synthetic and ground-truth images, ignoring the edge information, which is critical to provide clinical information. In this paper, we propose a novel edge-preserving MRI image synthesis method with iterative multi-scale feature fusion based generative adversarial network (EP_IMF-GAN). Particularly, the generator consists of a shared encoder and two specific decoders to carry out different tasks: 1) a primary task aiming to generate the target modality and 2) an auxiliary task aiming to generate the corresponding edge image of target modality. We assume that infusing the auxiliary edge image generation task can help preserve edge information and learn better latent representation features through the shared encoder. Meanwhile, an iterative multi-scale fusion module is embedded in the primary decoder to fuse supplementary information of feature maps at different scales, thereby further improving quality of the synthesized target modality. Experiments on the BRATS dataset indicate that our proposed method is superior to the state-of-the-art image synthesis approaches in both qualitative and quantitative measures. Ablation study further validates the effectiveness of the proposed components.
- Published
- 2021
10. S3Reg: Superfast Spherical Surface Registration Based on Deep Learning
- Author
-
Li Wang, Gang Li, Zhengwang Wu, Fenqiang Zhao, Fan Wang, Weili Lin, Dinggang Shen, and Shunren Xia
- Subjects
Adult ,Similarity (geometry) ,Computer science ,Neuroimaging ,Image processing ,Convolutional neural network ,Article ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,Kernel (linear algebra) ,Deep Learning ,0302 clinical medicine ,Image Processing, Computer-Assisted ,Humans ,Electrical and Electronic Engineering ,Quantitative Biology::Neurons and Cognition ,Radiological and Ultrasound Technology ,Artificial neural network ,business.industry ,Deep learning ,Infant ,Pattern recognition ,Magnetic Resonance Imaging ,Computer Science Applications ,Cross-Sectional Studies ,Feature (computer vision) ,Unsupervised learning ,Neural Networks, Computer ,Artificial intelligence ,business ,Software - Abstract
Cortical surface registration is an essential step and prerequisite for surface-based neuroimaging analysis. It aligns cortical surfaces across individuals and time points to establish cross-sectional and longitudinal cortical correspondences to facilitate neuroimaging studies. Though achieving good performance, available methods are either time consuming or not flexible to extend to multiple or high dimensional features. Considering the explosive availability of large-scale and multimodal brain MRI data, fast surface registration methods that can flexibly handle multimodal features are desired. In this study, we develop a Superfast Spherical Surface Registration (S3Reg) framework for the cerebral cortex. Leveraging an end-to-end unsupervised learning strategy, S3Reg offers great flexibility in the choice of input feature sets and output similarity measures for registration, and meanwhile reduces the registration time significantly. Specifically, we exploit the powerful learning capability of spherical Convolutional Neural Network (CNN) to directly learn the deformation fields in spherical space and implement diffeomorphic design with “scaling and squaring” layers to guarantee topology-preserving deformations. To handle the polar-distortion issue, we construct a novel spherical CNN model using three orthogonal Spherical U-Nets. Experiments are performed on two different datasets to align both adult and infant multimodal cortical features. Results demonstrate that our S3Reg shows superior or comparable performance with state-of-the-art methods, while improving the registration time from 1 min to 10 sec.
- Published
- 2021
11. iCOVID: interpretable deep learning framework for early recovery-time prediction of COVID-19 patients
- Author
-
Hongbing Lu, Jing Wang, B. Li, Dinggang Shen, Yaofeng Wen, X. Li, Chang Chen, Cheng Yuan, Cheng-Hui Liu, Y. Wang, Lichi Zhang, C. Jin, Jianrong Xu, Dong Qian, and Jianjun Li
- Subjects
2019-20 coronavirus outbreak ,medicine.medical_specialty ,Coronavirus disease 2019 (COVID-19) ,business.industry ,Deep learning ,Computer applications to medicine. Medical informatics ,Early recovery ,R858-859.7 ,Medicine (miscellaneous) ,Health Informatics ,Concordance index ,Article ,Computer Science Applications ,Text mining ,Health Information Management ,Viral infection ,Patient information ,Emergency medicine ,Machine learning ,medicine ,Mortality prediction ,Artificial intelligence ,business - Abstract
Most prior studies focused on developing models for the severity or mortality prediction of COVID-19 patients. However, effective models for recovery-time prediction are still lacking. Here, we present a deep learning solution named iCOVID that can successfully predict the recovery-time of COVID-19 patients based on predefined treatment schemes and heterogeneous multimodal patient information collected within 48 hours after admission. Meanwhile, an interpretable mechanism termed FSR is integrated into iCOVID to reveal the features greatly affecting the prediction of each patient. Data from a total of 3008 patients were collected from three hospitals in Wuhan, China, for large-scale verification. The experiments demonstrate that iCOVID can achieve a time-dependent concordance index of 74.9% (95% CI: 73.6-76.3%) and an average day error of 4.4 days (95% CI: 4.2-4.6 days). Our study reveals that treatment schemes, age, symptoms, comorbidities, and biomarkers are highly related to recovery-time predictions.
- Published
- 2021
12. HF-UNet: Learning Hierarchically Inter-Task Relevance in Multi-Task U-Net for Accurate Prostate Segmentation in CT Images
- Author
-
Yang Gao, Dinggang Shen, Junfeng Zhang, Xiaohuan Cao, Kelei He, Chunfeng Lian, Bing Zhang, Dong Nie, and Xin Zhang
- Subjects
Male ,Radiological and Ultrasound Technology ,Computer science ,business.industry ,Prostate ,Multi-task learning ,Pattern recognition ,Image segmentation ,Computer Science Applications ,Task (project management) ,Feature (computer vision) ,Image Processing, Computer-Assisted ,Task analysis ,Benchmark (computing) ,Humans ,Segmentation ,Artificial intelligence ,Electrical and Electronic Engineering ,Tomography, X-Ray Computed ,business ,Software ,Block (data storage) - Abstract
Accurate segmentation of the prostate is a key step in external beam radiation therapy treatments. In this paper, we tackle the challenging task of prostate segmentation in CT images by a two-stage network with 1) the first stage to fast localize, and 2) the second stage to accurately segment the prostate. To precisely segment the prostate in the second stage, we formulate prostate segmentation into a multi-task learning framework, which includes a main task to segment the prostate, and an auxiliary task to delineate the prostate boundary. Here, the second task is applied to provide additional guidance of unclear prostate boundary in CT images. Besides, the conventional multi-task deep networks typically share most of the parameters (i.e., feature representations) across all tasks, which may limit their data fitting ability, as the specificity of different tasks are inevitably ignored. By contrast, we solve them by a hierarchically-fused U-Net structure, namely HF-UNet. The HF-UNet has two complementary branches for two tasks, with the novel proposed attention-based task consistency learning block to communicate at each level between the two decoding branches. Therefore, HF-UNet endows the ability to learn hierarchically the shared representations for different tasks, and preserve the specificity of learned representations for different tasks simultaneously. We did extensive evaluations of the proposed method on a large planning CT image dataset and a benchmark prostate zonal dataset. The experimental results show HF-UNet outperforms the conventional multi-task network architectures and the state-of-the-art methods.
- Published
- 2021
13. Multi-Site Infant Brain Segmentation Algorithms: The iSeg-2019 Challenge
- Author
-
Li Zhao, Toan Duc Bui, Qi Dou, Yu Zhang, Sijie Niu, Trung Le Phan, Guannan Li, Longchuan Li, Sarah Shultz, Xiaopeng Zong, Wenao Ma, Gang Li, Yue Sun, Ying Wei, Xue Feng, Mallappa Kumara Swamy, Camilo Bermudez Noguera, Tao Zhong, Valerie Jewells, Li Wang, Weili Lin, Ramesh Basnet, Caizi Li, M. Omair Ahmad, Dinggang Shen, Zhihao Lei, Ian H. Gotlib, Kathryn L. Humphreys, Jun Ma, Bennett A. Landman, Jitae Shin, Kun Gao, Zhengwang Wu, Lequan Yu, and Xiaoping Yang
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,Brain development ,Computer science ,Computer Vision and Pattern Recognition (cs.CV) ,Computer Science - Computer Vision and Pattern Recognition ,Machine learning ,computer.software_genre ,Article ,Machine Learning (cs.LG) ,030218 nuclear medicine & medical imaging ,White matter ,03 medical and health sciences ,Consistency (database systems) ,0302 clinical medicine ,Cerebrospinal fluid ,FOS: Electrical engineering, electronic engineering, information engineering ,medicine ,Humans ,Brain segmentation ,Brain magnetic resonance imaging ,Gray Matter ,Electrical and Electronic Engineering ,Set (psychology) ,Brain Mapping ,Radiological and Ultrasound Technology ,business.industry ,Deep learning ,Image and Video Processing (eess.IV) ,Multi site ,Brain ,Infant ,Image segmentation ,Electrical Engineering and Systems Science - Image and Video Processing ,Magnetic Resonance Imaging ,Computer Science Applications ,medicine.anatomical_structure ,Artificial intelligence ,business ,computer ,Algorithms ,Software - Abstract
To better understand early brain development in health and disorder, it is critical to accurately segment infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Deep learning-based methods have achieved state-of-the-art performance; h owever, one of the major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners. To promote methodological development in the community, the iSeg-2019 challenge ( http://iseg2019.web.unc.edu ) provides a set of 6-month infant subjects from multiple sites with different protocols/scanners for the participating methods. T raining/validation subjects are from UNC (MAP) and testing subjects are from UNC/UMN (BCP), Stanford University, and Emory University. By the time of writing, there are 30 automatic segmentation methods participated in the iSeg-2019. In this article, 8 top-ranked methods were reviewed by detailing their pipelines/implementations, presenting experimental results, and evaluating performance across different sites in terms of whole brain, regions of interest, and gyral landmark curves. We further pointed out their limitations and possible directions for addressing the multi-site issue. We find that multi-site consistency is still an open issue. We hope that the multi-site dataset in the iSeg-2019 and this review article will attract more researchers to address the challenging and critical multi-site issue in practice.
- Published
- 2021
14. Cascaded MultiTask 3-D Fully Convolutional Networks for Pancreas Segmentation
- Author
-
Kelei He, Zhenshan Shi, Seong-Whan Lee, Dinggang Shen, Dengwang Li, Jie Xue, Ehsan Adeli, Dong Nie, Xiyu Liu, and Yuanjie Zheng
- Subjects
Computer science ,Multi-task learning ,02 engineering and technology ,030218 nuclear medicine & medical imaging ,Convolution ,03 medical and health sciences ,Consistency (database systems) ,Imaging, Three-Dimensional ,0302 clinical medicine ,Robustness (computer science) ,Pancreatic cancer ,Image Interpretation, Computer-Assisted ,0202 electrical engineering, electronic engineering, information engineering ,medicine ,Humans ,Segmentation ,Electrical and Electronic Engineering ,Pancreas ,business.industry ,Pattern recognition ,Image segmentation ,medicine.disease ,Computer Science Applications ,Pancreatic Neoplasms ,Human-Computer Interaction ,Task (computing) ,Control and Systems Engineering ,Feature (computer vision) ,020201 artificial intelligence & image processing ,Neural Networks, Computer ,Artificial intelligence ,business ,Software ,Information Systems - Abstract
Automatic pancreas segmentation is crucial to the diagnostic assessment of diabetes or pancreatic cancer. However, the relatively small size of the pancreas in the upper body, as well as large variations of its location and shape in retroperitoneum, make the segmentation task challenging. To alleviate these challenges, in this article, we propose a cascaded multitask 3-D fully convolution network (FCN) to automatically segment the pancreas. Our cascaded network is composed of two parts. The first part focuses on fast locating the region of the pancreas, and the second part uses a multitask FCN with dense connections to refine the segmentation map for fine voxel-wise segmentation. In particular, our multitask FCN with dense connections is implemented to simultaneously complete tasks of the voxel-wise segmentation and skeleton extraction from the pancreas. These two tasks are complementary, that is, the extracted skeleton provides rich information about the shape and size of the pancreas in retroperitoneum, which can boost the segmentation of pancreas. The multitask FCN is also designed to share the low- and mid-level features across the tasks. A feature consistency module is further introduced to enhance the connection and fusion of different levels of feature maps. Evaluations on two pancreas datasets demonstrate the robustness of our proposed method in correctly segmenting the pancreas in various settings. Our experimental results outperform both baseline and state-of-the-art methods. Moreover, the ablation study shows that our proposed parts/modules are critical for effective multitask learning.
- Published
- 2021
15. A Mutual Multi-Scale Triplet Graph Convolutional Network for Classification of Brain Disorders Using Functional or Structural Connectivity
- Author
-
Jing Sui, Yeerfan Jiaerken, Mingxia Liu, Na Luo, Erkun Yang, Mingliang Wang, Dinggang Shen, Dongren Yao, and Pew Thian Yap
- Subjects
Scale (ratio) ,Computer science ,Network topology ,Article ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,0302 clinical medicine ,Neuroimaging ,medicine ,Humans ,Electrical and Electronic Engineering ,Brain Diseases ,Radiological and Ultrasound Technology ,medicine.diagnostic_test ,Brain disorder diagnosis ,business.industry ,Brain ,Pattern recognition ,Magnetic Resonance Imaging ,Computer Science Applications ,Graph (abstract data type) ,Artificial intelligence ,business ,Functional magnetic resonance imaging ,Software ,Diffusion MRI - Abstract
Brain connectivity alterations associated with mental disorders have been widely reported in both functional MRI (fMRI) and diffusion MRI (dMRI). However, extracting useful information from the vast amount of information afforded by brain networks remains a great challenge. Capturing network topology, graph convolutional networks (GCNs) have demonstrated to be superior in learning network representations tailored for identifying specific brain disorders. Existing graph construction techniques generally rely on a specific brain parcellation to define regions-of-interest (ROIs) to construct networks, often limiting the analysis into a single spatial scale. In addition, most methods focus on the pairwise relationships between the ROIs and ignore high-order associations between subjects. In this letter, we propose a mutual multi-scale triplet graph convolutional network (MMTGCN) to analyze functional and structural connectivity for brain disorder diagnosis. We first employ several templates with different scales of ROI parcellation to construct coarse-to-fine brain connectivity networks for each subject. Then, a triplet GCN (TGCN) module is developed to learn functional/structural representations of brain connectivity networks at each scale, with the triplet relationship among subjects explicitly incorporated into the learning process. Finally, we propose a template mutual learning strategy to train different scale TGCNs collaboratively for disease classification. Experimental results on 1, 160 subjects from three datasets with fMRI or dMRI data demonstrate that our MMTGCN outperforms several state-of-the-art methods in identifying three types of brain disorders.
- Published
- 2021
16. Estimating Reference Shape Model for Personalized Surgical Reconstruction of Craniomaxillofacial Defects
- Author
-
Dinggang Shen, Pew Thian Yap, Chunfeng Lian, Jihua Zhu, Leonel Perez, Peng Yuan, Jaime Gateno, Kim-Han Thung, Li Wang, Hung-Ying Lin, Deqiang Xiao, Steve Guo Fang Shen, James J. Xia, and Hannah Deng
- Subjects
Facial trauma ,Facial bone ,Computer science ,0206 medical engineering ,Biomedical Engineering ,02 engineering and technology ,Iterative reconstruction ,Overfitting ,Surgical planning ,Article ,Imaging, Three-Dimensional ,Image Processing, Computer-Assisted ,medicine ,Humans ,Computer vision ,Models, Statistical ,business.industry ,Sparse approximation ,medicine.disease ,020601 biomedical engineering ,Face ,Face (geometry) ,Artificial intelligence ,Tomography, X-Ray Computed ,business ,Surface reconstruction - Abstract
Objective: To estimate a patient-specific reference bone shape model for a patient with craniomaxillofacial (CMF) defects due to facial trauma. Methods: We proposed an automatic facial bone shape estimation framework using pre-traumatic conventional portrait photos and post-traumatic head computed tomography (CT) scans via a 3D face reconstruction and a deformable shape model. Specifically, a three-dimensional (3D) face was first reconstructed from the patient's pre-traumatic portrait photos. Second, a correlation model between the skin and bone surfaces was constructed using a sparse representation based on the CT images of training normal subjects. Third, by feeding the reconstructed 3D face into the correlation model, an initial reference shape model was generated. In addition, we refined the initial estimation by applying non-rigid surface matching between the initially estimated shape and the patient's post-traumatic bone based on the adaptive-focus deformable shape model (AFDSM). Furthermore, a statistical shape model, built from the training normal subjects, was utilized to constrain the deformation process to avoid overfitting. Results and Conclusion: The proposed method was evaluated using both synthetic and real patient data. Experimental results show that the patient's abnormal facial bony structure can be recovered using our method, and the estimated reference shape model is considered clinically acceptable by an experienced CMF surgeon. Significance: The proposed method is more suitable to the complex CMF defects for CMF reconstructive surgical planning.
- Published
- 2021
17. Deep Bayesian Hashing With Center Prior for Multi-Modal Neuroimage Retrieval
- Author
-
Chunfeng Lian, Pew Thian Yap, Dinggang Shen, Mingxia Liu, Erkun Yang, Dongren Yao, and Cao Bing
- Subjects
Similarity (geometry) ,Databases, Factual ,Radiological and Ultrasound Technology ,Computer science ,business.industry ,Hash function ,Bayesian probability ,Bayes Theorem ,Machine learning ,computer.software_genre ,Article ,030218 nuclear medicine & medical imaging ,Computer Science Applications ,Visualization ,03 medical and health sciences ,0302 clinical medicine ,Discriminative model ,Artificial intelligence ,Electrical and Electronic Engineering ,Hamming space ,business ,Image retrieval ,computer ,Software - Abstract
Multi-modal neuroimage retrieval has greatly facilitated the efficiency and accuracy of decision making in clinical practice by providing physicians with previous cases (with visually similar neuroimages) and corresponding treatment records. However, existing methods for image retrieval usually fail when applied directly to multi-modal neuroimage databases, since neuroimages generally have smaller inter-class variation and larger inter-modal discrepancy compared to natural images. To this end, we propose a deep Bayesian hash learning framework, called CenterHash, which can map multi-modal data into a shared Hamming space and learn discriminative hash codes from imbalanced multi-modal neuroimages. The key idea to tackle the small inter-class variation and large inter-modal discrepancy is to learn a common center representation for similar neuroimages from different modalities and encourage hash codes to be explicitly close to their corresponding center representations. Specifically, we measure the similarity between hash codes and their corresponding center representations and treat it as a center prior in the proposed Bayesian learning framework. A weighted contrastive likelihood loss function is also developed to facilitate hash learning from imbalanced neuroimage pairs. Comprehensive empirical evidence shows that our method can generate effective hash codes and yield state-of-the-art performance in cross-modal retrieval on three multi-modal neuroimage datasets.
- Published
- 2021
18. ML-DSVM+: A meta-learning based deep SVM+ for computer-aided diagnosis
- Author
-
Xiangmin Han, Jun Wang, Shihui Ying, Jun Shi, and Dinggang Shen
- Subjects
Artificial Intelligence ,Signal Processing ,Computer Vision and Pattern Recognition ,Software - Published
- 2023
19. Breast Tumor Segmentation in DCE-MRI With Tumor Sensitive Synthesis
- Author
-
Shuai Wang, Kun Sun, Li Wang, Liangqiong Qu, Fuhua Yan, Qian Wang, and Dinggang Shen
- Subjects
Artificial Intelligence ,Computer Networks and Communications ,Software ,Computer Science Applications - Abstract
Segmenting breast tumors from dynamic contrast-enhanced magnetic resonance (DCE-MR) images is a critical step for early detection and diagnosis of breast cancer. However, variable shapes and sizes of breast tumors, as well as inhomogeneous background, make it challenging to accurately segment tumors in DCE-MR images. Therefore, in this article, we propose a novel tumor-sensitive synthesis module and demonstrate its usage after being integrated with tumor segmentation. To suppress false-positive segmentation with similar contrast enhancement characteristics to true breast tumors, our tumor-sensitive synthesis module can feedback differential loss of the true and false breast tumors. Thus, by following the tumor-sensitive synthesis module after the segmentation predictions, the false breast tumors with similar contrast enhancement characteristics to the true ones will be effectively reduced in the learned segmentation model. Moreover, the synthesis module also helps improve the boundary accuracy while inaccurate predictions near the boundary will lead to higher loss. For the evaluation, we build a very large-scale breast DCE-MR image dataset with 422 subjects from different patients, and conduct comprehensive experiments and comparisons with other algorithms to justify the effectiveness, adaptability, and robustness of our proposed method.
- Published
- 2021
20. Exploiting Sparse Self-Representation and Particle Swarm Optimization for CNN Compression
- Author
-
Sijie Niu, Kun Gao, Pengfei Ma, Xizhan Gao, Hui Zhao, Jiwen Dong, Yuehui Chen, and Dinggang Shen
- Subjects
Artificial Intelligence ,Computer Networks and Communications ,Software ,Computer Science Applications - Abstract
Structured pruning has received ever-increasing attention as a method for compressing convolutional neural networks. However, most existing methods directly prune the network structure according to the statistical information of the parameters. Besides, these methods differentiate the pruning rates only in each pruning stage or even use the same pruning rate across all layers, rather than using learnable parameters. In this article, we propose a network redundancy elimination approach guided by the pruned model. Our proposed method can easily tackle multiple architectures and is scalable to the deeper neural networks because of the use of joint optimization during the pruning procedure. More specifically, we first construct a sparse self-representation for the filters or neurons of the well-trained model, which is useful for analyzing the relationship among filters. Then, we employ particle swarm optimization to learn pruning rates in a layerwise manner according to the performance of the pruned model, which can determine optimal pruning rates with the best performance of the pruned model. Under this criterion, the proposed pruning approach can remove more parameters without undermining the performance of the model. Experimental results demonstrate the effectiveness of our proposed method on different datasets and different architectures. For example, it can reduce 58.1% FLOPs for ResNet50 on ImageNet with only a 1.6% top-five error increase and 44.1% FLOPs for FCN_ResNet50 on COCO2017 with a 3% error increase, outperforming most state-of-the-art methods.
- Published
- 2022
21. A fully automatic AI system for tooth and alveolar bone segmentation from cone-beam CT images
- Author
-
Zhiming Cui, Yu Fang, Lanzhuju Mei, Bojun Zhang, Bo Yu, Jiameng Liu, Caiwen Jiang, Yuhang Sun, Lei Ma, Jiawei Huang, Yang Liu, Yue Zhao, Chunfeng Lian, Zhongxiang Ding, Min Zhu, and Dinggang Shen
- Subjects
Multidisciplinary ,stomatognathic system ,Artificial Intelligence ,Image Processing, Computer-Assisted ,Humans ,General Physics and Astronomy ,General Chemistry ,Cone-Beam Computed Tomography ,Tooth ,General Biochemistry, Genetics and Molecular Biology ,Workflow - Abstract
Accurate delineation of individual teeth and alveolar bones from dental cone-beam CT (CBCT) images is an essential step in digital dentistry for precision dental healthcare. In this paper, we present an AI system for efficient, precise, and fully automatic segmentation of real-patient CBCT images. Our AI system is evaluated on the largest dataset so far, i.e., using a dataset of 4,215 patients (with 4,938 CBCT scans) from 15 different centers. This fully automatic AI system achieves a segmentation accuracy comparable to experienced radiologists (e.g., 0.5% improvement in terms of average Dice similarity coefficient), while significant improvement in efficiency (i.e., 500 times faster). In addition, it consistently obtains accurate results on the challenging cases with variable dental abnormalities, with the average Dice scores of 91.5% and 93.0% for tooth and alveolar bone segmentation. These results demonstrate its potential as a powerful system to boost clinical workflows of digital dentistry.
- Published
- 2022
22. Attention-Guided Hybrid Network for Dementia Diagnosis With Structural MR Images
- Author
-
Yongsheng Pan, Mingxia Liu, Dinggang Shen, and Chunfeng Lian
- Subjects
Computer science ,Neuroimaging ,Convolutional neural network ,Structural magnetic resonance imaging ,Article ,Discriminative model ,Alzheimer Disease ,Humans ,Dementia diagnosis ,Attention ,Cognitive Dysfunction ,Electrical and Electronic Engineering ,Cognitive impairment ,business.industry ,Prodromal Stage ,Brain ,Pattern recognition ,Magnetic Resonance Imaging ,Computer Science Applications ,Human-Computer Interaction ,Control and Systems Engineering ,Artificial intelligence ,Mr images ,business ,Software ,Information Systems - Abstract
Deep-learning methods (especially convolutional neural networks) using structural magnetic resonance imaging (sMRI) data have been successfully applied to computer-aided diagnosis (CAD) of Alzheimer's disease (AD) and its prodromal stage [i.e., mild cognitive impairment (MCI)]. As it is practically challenging to capture local and subtle disease-associated abnormalities directly from the whole-brain sMRI, most of those deep-learning approaches empirically preselect disease-associated sMRI brain regions for model construction. Considering that such isolated selection of potentially informative brain locations might be suboptimal, very few methods have been proposed to perform disease-associated discriminative region localization and disease diagnosis in a unified deep-learning framework. However, those methods based on task-oriented discriminative localization still suffer from two common limitations, that is: 1) identified brain locations are strictly consistent across all subjects, which ignores the unique anatomical characteristics of each brain and 2) only limited local regions/patches are used for model training, which does not fully utilize the global structural information provided by the whole-brain sMRI. In this article, we propose an attention-guided deep-learning framework to extract multilevel discriminative sMRI features for dementia diagnosis. Specifically, we first design a backbone fully convolutional network to automatically localize the discriminative brain regions in a weakly supervised manner. Using the identified disease-related regions as spatial attention guidance, we further develop a hybrid network to jointly learn and fuse multilevel sMRI features for CAD model construction. Our proposed method was evaluated on three public datasets (i.e., ADNI-1, ADNI-2, and AIBL), showing superior performance compared with several state-of-the-art methods in both tasks of AD diagnosis and MCI conversion prediction.
- Published
- 2022
23. Deep Learning and Medical Image Analysis for COVID-19 Diagnosis and Prediction
- Author
-
Tianming Liu, Eliot Siegel, and Dinggang Shen
- Subjects
COVID-19 Testing ,Deep Learning ,Artificial Intelligence ,SARS-CoV-2 ,Biomedical Engineering ,Medicine (miscellaneous) ,COVID-19 ,Humans - Abstract
The coronavirus disease 2019 (COVID-19) pandemic has imposed dramatic challenges to health-care organizations worldwide. To combat the global crisis, the use of thoracic imaging has played a major role in the diagnosis, prediction, and management of COVID-19 patients with moderate to severe symptoms or with evidence of worsening respiratory status. In response, the medical image analysis community acted quickly to develop and disseminate deep learning models and tools to meet the urgent need of managing and interpreting large amounts of COVID-19 imaging data. This review aims to not only summarize existing deep learning and medical image analysis methods but also offer in-depth discussions and recommendations for future investigations. We believe that the wide availability of high-quality, curated, and benchmarked COVID-19 imaging data sets offers the great promise of a transformative test bed to develop, validate, and disseminate novel deep learning methods in the frontiers of data science and artificial intelligence.
- Published
- 2022
24. Hierarchical Nonlocal Residual Networks for Image Quality Assessment of Pediatric Diffusion MRI with Limited and Noisy Annotations
- Author
-
Pew Thian Yap, Kim-Han Thung, Dinggang Shen, Weili Lin, and Siyuan Liu
- Subjects
semi-supervised learning ,hierarchical nonlocal residual networks ,Image quality ,Computer science ,Feature extraction ,Residual ,Article ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,0302 clinical medicine ,Image quality assessment ,self-training ,Image Processing, Computer-Assisted ,Humans ,Electrical and Electronic Engineering ,Child ,Radiological and Ultrasound Technology ,Noise measurement ,business.industry ,Deep learning ,Pattern recognition ,Computer Science Applications ,Diffusion Magnetic Resonance Imaging ,Artificial intelligence ,business ,Software ,Volume (compression) ,Diffusion MRI - Abstract
Fast and automated image quality assessment (IQA) for diffusion MR images is a crucial step for swiftly making a rescan decision during or after the scanning session. However, learning a model for this task is challenging as the number of annotated data is limited and the annotation labels might not always be correct. As a remedy, we will introduce in this paper an automatic image quality assessment (IQA) method based on hierarchical non-local residual networks for pediatric diffusion MR images. Our IQA is performed in three sequential stages, i.e., 1) slice-wise IQA, where a nonlocal residual network is first pre-trained to annotate each slice with an initial quality rating (i.e., pass/questionable/fail), which is subsequently refined via iterative semi-supervised learning and slice self-training; 2) volume-wise IQA, which agglomerates the features extracted from slices of a volume, and uses a nonlocal network to annotate the quality rating for each volume via iterative volume self-training; and 3) subject-wise IQA, which ensembles the volumetric IQA results to determine the overall image quality pertaining to a subject. Experimental results demonstrate that our method, trained using only samples of modest size, exhibits great generalizability, and is capable of conducting rapid hierarchical IQA with near-perfect accuracy.
- Published
- 2020
25. Multi-Class ASD Classification Based on Functional Connectivity and Functional Correlation Tensor via Multi-Source Domain Adaptation and Multi-View Sparse Representation
- Author
-
Zuoyong Li, Qian Wang, Xiaobo Chen, Lichi Zhang, Lei Chen, Jun Shi, Jun Wang, and Dinggang Shen
- Subjects
Autism Spectrum Disorder ,Computer science ,Feature vector ,Feature extraction ,Grey matter ,030218 nuclear medicine & medical imaging ,White matter ,03 medical and health sciences ,0302 clinical medicine ,medicine ,Humans ,Gray Matter ,Electrical and Electronic Engineering ,Brain Mapping ,Radiological and Ultrasound Technology ,medicine.diagnostic_test ,business.industry ,Functional connectivity ,Brain ,Pattern recognition ,Sparse approximation ,medicine.disease ,Magnetic Resonance Imaging ,Computer Science Applications ,medicine.anatomical_structure ,Autism spectrum disorder ,Artificial intelligence ,Functional magnetic resonance imaging ,business ,Classifier (UML) ,Software ,Multi-source - Abstract
The resting-state functional magnetic resonance imaging (rs-fMRI) reflects functional activity of brain regions by blood-oxygen-level dependent (BOLD) signals. Up to now, many computer-aided diagnosis methods based on rs-fMRI have been developed for Autism Spectrum Disorder (ASD). These methods are mostly the binary classification approaches to determine whether a subject is an ASD patient or not. However, the disease often consists of several sub-categories, which are complex and thus still confusing to many automatic classification methods. Besides, existing methods usually focus on the functional connectivity (FC) features in grey matter regions, which only account for a small portion of the rs-fMRI data. Recently, the possibility to reveal the connectivity information in the white matter regions of rs-fMRI has drawn high attention. To this end, we propose to use the patch-based functional correlation tensor (PBFCT) features extracted from rs-fMRI in white matter, in addition to the traditional FC features from gray matter, to develop a novel multi-class ASD diagnosis method in this work. Our method has two stages. Specifically, in the first stage of multi-source domain adaptation (MSDA), the source subjects belonging to multiple clinical centers (thus called as source domains) are all transformed into the same target feature space. Thus each subject in the target domain can be linearly reconstructed by the transformed subjects. In the second stage of multi-view sparse representation (MVSR), a multi-view classifier for multi-class ASD diagnosis is developed by jointly using both views of the FC and PBFCT features. The experimental results using the ABIDE dataset verify the effectiveness of our method, which is capable of accurately classifying each subject into a respective ASD sub-category.
- Published
- 2020
26. Iterative Label Denoising Network: Segmenting Male Pelvic Organs in CT From 3D Bounding Box Annotations
- Author
-
Jun Lian, Qian Wang, Shuai Wang, Liangqiong Qu, Dinggang Shen, Yeqin Shao, and Chunfeng Lian
- Subjects
Male ,Similarity (geometry) ,Computer science ,Urinary Bladder ,0206 medical engineering ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Biomedical Engineering ,02 engineering and technology ,computer.software_genre ,Article ,Market segmentation ,Voxel ,Minimum bounding box ,Medical imaging ,Humans ,Segmentation ,business.industry ,Supervised learning ,Prostatic Neoplasms ,Pattern recognition ,Image segmentation ,020601 biomedical engineering ,Artificial intelligence ,Tomography, X-Ray Computed ,business ,computer ,Algorithms - Abstract
Obtaining accurate segmentation of the prostate and nearby organs at risk (e.g., bladder and rectum) in CT images is critical for radiotherapy of prostate cancer. Currently, the leading automatic segmentation algorithms are based on Fully Convolutional Networks (FCNs), which achieve remarkable performance but usually need large-scale datasets with high-quality voxel-wise annotations for full supervision of the training. Unfortunately, such annotations are difficult to acquire, which becomes a bottleneck to build accurate segmentation models in real clinical applications. In this paper, we propose a novel weakly supervised segmentation approach that only needs 3D bounding box annotations covering the organs of interest to start the training. Obviously, the bounding box includes many non-organ voxels that carry noisy labels to mislead the segmentation model. To this end, we propose the label denoising module and embed it into the iterative training scheme of the label denoising network (LDnet) for segmentation. The labels of the training voxels are predicted by the tentative LDnet, while the label denoising module identifies the voxels with unreliable labels. As only the good training voxels are preserved, the iteratively re-trained LDnet can refine its segmentation capability gradually. Our results are remarkable, i.e., reaching $\sim$ 94% (prostate), $\sim$ 91% (bladder), and $\sim$ 86% (rectum) of the Dice Similarity Coefficients (DSCs), compared to the case of fully supervised learning upon high-quality voxel-wise annotations and also superior to several state-of-the-art approaches. To our best knowledge, this is the first work to achieve voxel-wise segmentation in CT images from simple 3D bounding box annotations, which can greatly reduce many labeling efforts and meet the demands of the practical clinical applications.
- Published
- 2020
27. Multi-View Spatial Aggregation Framework for Joint Localization and Segmentation of Organs at Risk in Head and Neck CT Images
- Author
-
Dong Nie, Yu Zhang, Dinggang Shen, Kim-Han Thung, and Shujun Liang
- Subjects
Organs at Risk ,Computer science ,Image processing ,Convolutional neural network ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,0302 clinical medicine ,Image Processing, Computer-Assisted ,Humans ,Segmentation ,Electrical and Electronic Engineering ,Representation (mathematics) ,Radiological and Ultrasound Technology ,Artificial neural network ,business.industry ,Deep learning ,Pattern recognition ,Image segmentation ,Computer Science Applications ,Head and Neck Neoplasms ,Neural Networks, Computer ,Tomography ,Artificial intelligence ,Tomography, X-Ray Computed ,business ,Software - Abstract
Accurate segmentation of organs at risk (OARs) from head and neck (H&N) CT images is crucial for effective H&N cancer radiotherapy. However, the existing deep learning methods are often not trained in an end-to-end fashion, i.e., they independently predetermine the regions of target organs before organ segmentation, causing limited information sharing between related tasks and thus leading to suboptimal segmentation results. Furthermore, when conventional segmentation network is used to segment all the OARs simultaneously, the results often favor big OARs over small OARs. Thus, the existing methods often train a specific model for each OAR, ignoring the correlation between different segmentation tasks. To address these issues, we propose a new multi-view spatial aggregation framework for joint localization and segmentation of multiple OARs using H&N CT images. The core of our framework is a proposed region-of-interest (ROI)-based fine-grained representation convolutional neural network (CNN), which is used to generate multi-OAR probability maps from each 2D view (i.e., axial, coronal, and sagittal view) of CT images. Specifically, our ROI-based fine-grained representation CNN (1) unifies the OARs localization and segmentation tasks and trains them in an end-to-end fashion, and (2) improves the segmentation results of various-sized OARs via a novel ROI-based fine-grained representation. Our multi-view spatial aggregation framework then spatially aggregates and assembles the generated multi-view multi-OAR probability maps to segment all the OARs simultaneously. We evaluate our framework using two sets of H&N CT images and achieve competitive and highly robust segmentation performance for OARs of various sizes.
- Published
- 2020
28. Spatially-Constrained Fisher Representation for Brain Disease Identification With Incomplete Multi-Modal Neuroimages
- Author
-
Yong Xia, Yongsheng Pan, Dinggang Shen, Mingxia Liu, and Chunfeng Lian
- Subjects
Computer science ,Feature extraction ,Article ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,0302 clinical medicine ,medicine ,Humans ,Imputation (statistics) ,Electrical and Electronic Engineering ,Medical diagnosis ,Brain Diseases ,Radiological and Ultrasound Technology ,medicine.diagnostic_test ,business.industry ,Deep learning ,Brain ,Pattern recognition ,Magnetic resonance imaging ,Mixture model ,Magnetic Resonance Imaging ,Computer Science Applications ,Brain disease ,Constraint (information theory) ,Identification (information) ,Positron emission tomography ,Positron-Emission Tomography ,Artificial intelligence ,Tomography, X-Ray Computed ,business ,Software - Abstract
Multi-modal neuroimages, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), can provide complementary structural and functional information of the brain, thus facilitating automated brain disease identification. Incomplete data problem is unavoidable in multi-modal neuroimage studies due to patient dropouts and/or poor data quality. Conventional methods usually discard data-missing subjects, thus significantly reducing the number of training samples. Even though several deep learning methods have been proposed, they usually rely on pre-defined regions-of-interest in neuroimages, requiring disease-specific expert knowledge. To this end, we propose a spatially-constrained Fisher representation framework for brain disease diagnosis with incomplete multi-modal neuroimages. We first impute missing PET images based on their corresponding MRI scans using a hybrid generative adversarial network. With the complete (after imputation) MRI and PET data, we then develop a spatially-constrained Fisher representation network to extract statistical descriptors of neuroimages for disease diagnosis, assuming that these descriptors follow a Gaussian mixture model with a strong spatial constraint (i.e., images from different subjects have similar anatomical structures). Experimental results on three databases suggest that our method can synthesize reasonable neuroimages and achieve promising results in brain disease identification, compared with several state-of-the-art methods.
- Published
- 2020
29. An Effective MR-Guided CT Network Training for Segmenting Prostate in CT Images
- Author
-
Sang-Hyun Park, Yinghuan Shi, Yang Gao, Ming Yang, Wanqi Yang, and Dinggang Shen
- Subjects
Male ,Computer science ,02 engineering and technology ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,Deep Learning ,0302 clinical medicine ,Health Information Management ,Prostate ,Image Processing, Computer-Assisted ,0202 electrical engineering, electronic engineering, information engineering ,Medical imaging ,medicine ,Humans ,Segmentation ,Computer vision ,Electrical and Electronic Engineering ,Ground truth ,medicine.diagnostic_test ,business.industry ,Magnetic resonance imaging ,Image segmentation ,Magnetic Resonance Imaging ,Computer Science Applications ,medicine.anatomical_structure ,020201 artificial intelligence & image processing ,Artificial intelligence ,Tomography, X-Ray Computed ,business ,Mri guided ,Biotechnology ,Prostate segmentation - Abstract
Segmentation of prostate in medical imaging data (e.g., CT, MRI, TRUS) is often considered as a critical yet challenging task for radiotherapy treatment. It is relatively easier to segment prostate from MR images than from CT images, due to better soft tissue contrast of the MR images. For segmenting prostate from CT images, most previous methods mainly used CT alone, and thus their performances are often limited by low tissue contrast in the CT images. In this article, we explore the possibility of using indirect guidance from MR images for improving prostate segmentation in the CT images. In particular, we propose a novel deep transfer learning approach, i.e., MR-guided CT network training (namely MICS-NET), which can employ MR images to help better learning of features in CT images for prostate segmentation. In MICS-NET, the guidance from MRI consists of two steps: (1) learning informative and transferable features from MRI and then transferring them to CT images in a cascade manner, and (2) adaptively transferring the prostate likelihood of MRI model (i.e., well-trained convnet by purely using MR images) with a view consistency constraint. To illustrate the effectiveness of our approach, we evaluate MICS-NET on a real CT prostate image set, with the manual delineations available as the ground truth for evaluation. Our methods generate promising segmentation results which achieve (1) six percentages higher Dice Ratio than the CT model purely using CT images and (2) comparable performance with the MRI model purely using MR images.
- Published
- 2020
30. Spatial-Temporal Dependency Modeling and Network Hub Detection for Functional MRI Analysis via Convolutional-Recurrent Network
- Author
-
Mingliang Wang, Daoqiang Zhang, Dinggang Shen, Dongren Yao, Chunfeng Lian, and Mingxia Liu
- Subjects
Dependency (UML) ,Computer science ,0206 medical engineering ,Biomedical Engineering ,Neuroimaging ,02 engineering and technology ,Timely diagnosis ,Article ,Discriminative model ,Alzheimer Disease ,Sliding window protocol ,medicine ,Humans ,Dementia ,Cognitive Dysfunction ,Cognitive impairment ,medicine.diagnostic_test ,Artificial neural network ,business.industry ,Deep learning ,Pattern recognition ,medicine.disease ,Magnetic Resonance Imaging ,020601 biomedical engineering ,Identification (information) ,Neural Networks, Computer ,Artificial intelligence ,Functional magnetic resonance imaging ,business - Abstract
Early identification of dementia at the stage of mild cognitive impairment (MCI) is crucial for timely diagnosis and intervention of Alzheimer's disease (AD). Although several pioneering studies have been devoted to automated AD diagnosis based on resting-state functional magnetic resonance imaging (rs-fMRI), their performance is somewhat limited due to non-effective mining of spatial-temporal dependency. Besides, few of these existing approaches consider the explicit detection and modeling of discriminative brain regions (i.e., network hubs) that are sensitive to AD progression. In this paper, we propose a unique Spatial-Temporal convolutional-recurrent neural Network (STNet) for automated prediction of AD progression and network hub detection from rs-fMRI time series. Our STNet incorporates the spatial-temporal information mining and AD-related hub detection into an end-to-end deep learning model. Specifically, we first partition rs-fMRI time series into a sequence of overlapping sliding windows. A sequence of convolutional components are then designed to capture the local-to-global spatially-dependent patterns within each sliding window, based on which we are able to identify discriminative hubs and characterize their unique contributions to disease diagnosis. A recurrent component with long short-term memory (LSTM) units is further employed to model the whole-brain temporal dependency from the spatially-dependent pattern sequences, thus capturing the temporal dynamics along time. We evaluate the proposed method on 174 subjects with 563 rs-fMRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, with results suggesting the effectiveness of our method in both tasks of disease progression prediction and AD-related hub detection.
- Published
- 2020
31. Modeling Hierarchical Brain Networks via Volumetric Sparse Deep Belief Network
- Author
-
Xi Jiang, Jing Yuan, Fangfei Ge, Yu Zhao, Tianming Liu, Qiang Ning, Jinglei Lv, Heng Huang, Qinglin Dong, and Dinggang Shen
- Subjects
Human Connectome Project ,medicine.diagnostic_test ,Computer science ,business.industry ,Deep learning ,0206 medical engineering ,Biomedical Engineering ,Brain ,02 engineering and technology ,Machine learning ,computer.software_genre ,Magnetic Resonance Imaging ,020601 biomedical engineering ,Convolutional neural network ,Deep belief network ,Recurrent neural network ,Connectome ,medicine ,Humans ,Neural Networks, Computer ,Artificial intelligence ,Functional magnetic resonance imaging ,business ,computer ,Software - Abstract
It has been recently shown that deep learning models such as convolutional neural networks (CNN), deep belief networks (DBN) and recurrent neural networks (RNN), exhibited remarkable ability in modeling and representing fMRI data for the understanding of functional activities and networks because of their superior data representation capability and wide availability of effective deep learning tools. For example, spatial and/or temporal patterns of functional brain activities embedded in fMRI data can be effectively characterized and modeled by a variety of CNN/DBN/RNN deep learning models as shown in recent studies. However, it has been rarely investigated whether it is possible to directly infer hierarchical brain networks from volumetric fMRI data using deep learning models such as DBN. The perceived difficulties of such studies include very large number of input variables, very large number of training parameters, the lack of effective software tools, the challenge of results interpretation, and etc. To bridge these technical gaps, we designed a novel volumetric sparse deep belief network (VS-DBN) model and implemented it through the popular TensorFlow open source platform to reconstruct hierarchical brain networks from volumetric fMRI data based on the Human Connectome Project (HCP) 900 subjects release. Our experimental results showed that a large number of interpretable and meaningful brain networks can be robustly reconstructed from HCP 900 subjects in a hierarchical fashion, and importantly, these brain networks exhibit reasonably good consistency and correspondence across multiple HCP task-based fMRI datasets. Our work contributed a new general deep learning framework for inferring multiscale volumetric brain networks and offered novel insights into the hierarchical organization of functional brain architecture.
- Published
- 2020
32. Learning longitudinal classification-regression model for infant hippocampus segmentation
- Author
-
Dinggang Shen, Zhengwang Wu, and Yanrong Guo
- Subjects
0209 industrial biotechnology ,Brain development ,Computer science ,Cognitive Neuroscience ,Hippocampus ,Context (language use) ,02 engineering and technology ,Spatial memory ,020901 industrial engineering & automation ,Artificial Intelligence ,0202 electrical engineering, electronic engineering, information engineering ,medicine ,Hippocampus (mythology) ,Segmentation ,medicine.diagnostic_test ,business.industry ,Contrast (statistics) ,Magnetic resonance imaging ,Pattern recognition ,Human brain ,Computer Science Applications ,medicine.anatomical_structure ,Hippocampal volume ,020201 artificial intelligence & image processing ,Artificial intelligence ,business - Abstract
Hippocampus plays an important role in the memory and spatial navigation function of human brain. Study on its growth and change during first year of life would assist the investigation of early brain development as well as the biomarker for neurological disorders. With the help of Magnetic Resonance (MR) imaging techniques, infant brain at different development stage can be acquired with multiple imaging modalities. In this situation, the longitudinal segmentation of infant hippocampus is highly demanded and feasible for the clinical studies regarding to the hippocampal volume changes. However, since the brain structures undergo dynamic appearance, structural changes and various tissue contrast during the first year of life, substantial challenges will be imposed for ensuring the robustness and accuracy of automatic hippocampus segmentation algorithms. In addition, most of the existing hippocampus segmentation methods generally handle each brain development stage independently without considering the potential longitudinal consistency among different stages. In view of the above factors, we propose a longitudinal classification-regression model for segmenting hippocampus in infant brain MRIs. Generally, our model proceeds on a per-timepoint basis, guided by the output of latter timepoint towards the infant hippocampus in the previous timepoint. The key ingredient of our method is a combination of longitudinal context, static context and appearance learning strategies under the classification-regression forest architecture. Specifically, the longitudinal context is borrowed from the mask of prior-timepoint estimation and the static context is from the current-timepoint estimation. Furthermore, we implement the proposed model in a multi-scale and iterative manner to improve the efficiency and effectiveness. The proposed method is evaluated on segmenting infant hippocampi from T1-weighted brain MR images acquired at the age of 2 weeks, 3 months, 6 months, 9 months, and 12 months. Experimental results demonstrate that our method achieves better performance in segmentation accuracy over the state-of-the-art classification and regression random forest model.
- Published
- 2020
33. Non-Local U-Nets for Biomedical Image Segmentation
- Author
-
Shuiwang Ji, Zhengyang Wang, Na Zou, and Dinggang Shen
- Subjects
Computer science ,business.industry ,Deep learning ,Aggregate (data warehouse) ,Biomedical image ,020207 software engineering ,02 engineering and technology ,General Medicine ,Non local ,Task (computing) ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Segmentation ,Computer vision ,Artificial intelligence ,business - Abstract
Deep learning has shown its great promise in various biomedical image segmentation tasks. Existing models are typically based on U-Net and rely on an encoder-decoder architecture with stacked local operators to aggregate long-range information gradually. However, only using the local operators limits the efficiency and effectiveness. In this work, we propose the non-local U-Nets, which are equipped with flexible global aggregation blocks, for biomedical image segmentation. These blocks can be inserted into U-Net as size-preserving processes, as well as down-sampling and up-sampling layers. We perform thorough experiments on the 3D multimodality isointense infant brain MR image segmentation task to evaluate the non-local U-Nets. Results show that our proposed models achieve top performances with fewer parameters and faster computation.
- Published
- 2020
34. Auto-GAN: Self-Supervised Collaborative Learning for Medical Image Synthesis
- Author
-
Han Zhang, Cao Bing, Xinbo Gao, Dinggang Shen, and Nannan Wang
- Subjects
0303 health sciences ,Modality (human–computer interaction) ,genetic structures ,Computer science ,business.industry ,Generalization ,0206 medical engineering ,Collaborative learning ,02 engineering and technology ,General Medicine ,Machine learning ,computer.software_genre ,Missing data ,Image (mathematics) ,Constraint (information theory) ,03 medical and health sciences ,otorhinolaryngologic diseases ,Artificial intelligence ,business ,computer ,psychological phenomena and processes ,020602 bioinformatics ,030304 developmental biology ,Generator (mathematics) - Abstract
In various clinical scenarios, medical image is crucial in disease diagnosis and treatment. Different modalities of medical images provide complementary information and jointly helps doctors to make accurate clinical decision. However, due to clinical and practical restrictions, certain imaging modalities may be unavailable nor complete. To impute missing data with adequate clinical accuracy, here we propose a framework called self-supervised collaborative learning to synthesize missing modality for medical images. The proposed method comprehensively utilize all available information correlated to the target modality from multi-source-modality images to generate any missing modality in a single model. Different from the existing methods, we introduce an auto-encoder network as a novel, self-supervised constraint, which provides target-modality-specific information to guide generator training. In addition, we design a modality mask vector as the target modality label. With experiments on multiple medical image databases, we demonstrate a great generalization ability as well as specialty of our method compared with other state-of-the-arts.
- Published
- 2020
35. Hierarchical Fully Convolutional Network for Joint Atrophy Localization and Alzheimer's Disease Diagnosis Using Structural MRI
- Author
-
Mingxia Liu, Jun Zhang, Dinggang Shen, and Chunfeng Lian
- Subjects
Male ,Computer science ,Feature extraction ,02 engineering and technology ,Convolutional neural network ,Article ,Atrophy ,Discriminative model ,Alzheimer Disease ,Artificial Intelligence ,Image Interpretation, Computer-Assisted ,0202 electrical engineering, electronic engineering, information engineering ,medicine ,Humans ,Medical diagnosis ,Aged ,Aged, 80 and over ,Contextual image classification ,business.industry ,Applied Mathematics ,Deep learning ,Brain ,Pattern recognition ,medicine.disease ,Magnetic Resonance Imaging ,Support vector machine ,Computational Theory and Mathematics ,Computer-aided diagnosis ,Female ,020201 artificial intelligence & image processing ,Neural Networks, Computer ,Supervised Machine Learning ,Computer Vision and Pattern Recognition ,Artificial intelligence ,business ,Software - Abstract
Structural magnetic resonance imaging (sMRI) has been widely used for computer-aided diagnosis of neurodegenerative disorders, e.g., Alzheimer’s disease (AD), due to its sensitivity to morphological changes caused by brain atrophy. Recently, a few deep learning methods (e.g., convolutional neural networks, CNNs) have been proposed to learn task-oriented features from sMRI for AD diagnosis, and achieved superior performance than the conventional learning-based methods using hand-crafted features. However, these existing CNN-based methods still require the pre-determination of informative locations in sMRI. That is, the stage of discriminative atrophy localization is isolated to the latter stages of feature extraction and classifier construction. In this paper, we propose a hierarchical fully convolutional network (H-FCN) to automatically identify discriminative local patches and regions in the whole brain sMRI, upon which multi-scale feature representations are then jointly learned and fused to construct hierarchical classification models for AD diagnosis. Our proposed H-FCN method was evaluated on a large cohort of subjects from two independent datasets (i.e., ADNI-1 and ADNI-2), demonstrating good performance on joint discriminative atrophy localization and brain disease diagnosis.
- Published
- 2020
36. Mammographic mass segmentation using multichannel and multiscale fully convolutional networks
- Author
-
Jie-Zhi Cheng, Lei Xiang, Seong-Whan Lee, Yang Li, Shengzhou Xu, Ehsan Adeli, and Dinggang Shen
- Subjects
Computer science ,business.industry ,Pattern recognition ,Computer Vision and Pattern Recognition ,Mass segmentation ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,Software ,Electronic, Optical and Magnetic Materials - Published
- 2020
37. One-Shot Generative Adversarial Learning for MRI Segmentation of Craniomaxillofacial Bony Structures
- Author
-
James J. Xia, Xu Chen, Dong Nie, Chunfeng Lian, Jaime Gateno, Dinggang Shen, Li Wang, Pew Thian Yap, Kim-Han Thung, Steve H. Fung, and Hannah Deng
- Subjects
Computer science ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Computed tomography ,Facial Bones ,Article ,030218 nuclear medicine & medical imaging ,Machine Learning ,03 medical and health sciences ,0302 clinical medicine ,Image Interpretation, Computer-Assisted ,Image Processing, Computer-Assisted ,medicine ,Humans ,Electrical and Electronic Engineering ,One shot ,Radiological and Ultrasound Technology ,medicine.diagnostic_test ,business.industry ,Skull ,Magnetic resonance imaging ,Pattern recognition ,Magnetic Resonance Imaging ,Computer Science Applications ,Radiation exposure ,Neural Networks, Computer ,Artificial intelligence ,Tomography, X-Ray Computed ,business ,Software ,Mri segmentation ,Generative grammar - Abstract
Compared to computed tomography (CT), magnetic resonance imaging (MRI) delineation of craniomaxillofacial (CMF) bony structures can avoid harmful radiation exposure. However, bony boundaries are blurry in MRI, and structural information needs to be borrowed from CT during the training. This is challenging since paired MRI-CT data are typically scarce. In this paper, we propose to make full use of unpaired data, which are typically abundant, along with a single paired MRI-CT data to construct a one-shot generative adversarial model for automated MRI segmentation of CMF bony structures. Our model consists of a cross-modality image synthesis sub-network , which learns the mapping between CT and MRI, and an MRI segmentation sub-network . These two sub-networks are trained jointly in an end-to-end manner. Moreover, in the training phase, a neighbor-based anchoring method is proposed to reduce the ambiguity problem inherent in cross-modality synthesis, and a feature-matching-based semantic consistency constraint is proposed to encourage segmentation-oriented MRI synthesis. Experimental results demonstrate the superiority of our method both qualitatively and quantitatively in comparison with the state-of-the-art MRI segmentation methods.
- Published
- 2020
38. A toolbox for brain network construction and classification (BrainNetClass)
- Author
-
Han Zhang, Xiaobo Chen, Renping Yu, Zhen Zhou, Dan Hu, Dinggang Shen, Yu Zhang, Pew Thian Yap, Gang Pan, and Lishan Qiao
- Subjects
Computer science ,brain connectome ,Machine learning ,computer.software_genre ,050105 experimental psychology ,Field (computer science) ,03 medical and health sciences ,0302 clinical medicine ,Robustness (computer science) ,toolbox ,Connectome ,Image Processing, Computer-Assisted ,Humans ,0501 psychology and cognitive sciences ,Radiology, Nuclear Medicine and imaging ,sparse representation ,Research Articles ,Network model ,Dynamic functional connectivity ,Interpretability ,Radiological and Ultrasound Technology ,business.industry ,05 social sciences ,functional connectivity ,Brain ,prediction ,Magnetic Resonance Imaging ,Toolbox ,machine learning ,Neurology ,Pairwise comparison ,dynamic functional connectivity ,Neurology (clinical) ,Artificial intelligence ,Anatomy ,Nerve Net ,business ,computer ,030217 neurology & neurosurgery ,Software ,Research Article - Abstract
Brain functional network has been increasingly used in understanding brain functions and diseases. While many network construction methods have been proposed, the progress in the field still largely relies on static pairwise Pearson's correlation‐based functional network and group‐level comparisons. We introduce a “Brain Network Construction and Classification (BrainNetClass)” toolbox to promote more advanced brain network construction methods to the filed, including some state‐of‐the‐art methods that were recently developed to capture complex and high‐order interactions among brain regions. The toolbox also integrates a well‐accepted and rigorous classification framework based on brain connectome features toward individualized disease diagnosis in a hope that the advanced network modeling could boost the subsequent classification. BrainNetClass is a MATLAB‐based, open‐source, cross‐platform toolbox with both graphical user‐friendly interfaces and a command line mode targeting cognitive neuroscientists and clinicians for promoting reliability, reproducibility, and interpretability of connectome‐based, computer‐aided diagnosis. It generates abundant classification‐related results from network presentations to contributing features that have been largely ignored by most studies to grant users the ability of evaluating the disease diagnostic model and its robustness and generalizability. We demonstrate the effectiveness of the toolbox on real resting‐state functional MRI datasets. BrainNetClass (v1.0) is available at https://github.com/zzstefan/BrainNetClass.
- Published
- 2020
39. Identifying Autism Spectrum Disorder With Multi-Site fMRI via Low-Rank Domain Adaptation
- Author
-
Dinggang Shen, Jiashuang Huang, Pew Thian Yap, Mingxia Liu, Daoqiang Zhang, and Mingliang Wang
- Subjects
Autism Spectrum Disorder ,Computer science ,Article ,Statistical power ,030218 nuclear medicine & medical imaging ,Data modeling ,Domain (software engineering) ,03 medical and health sciences ,0302 clinical medicine ,Neurodevelopmental disorder ,Image Processing, Computer-Assisted ,medicine ,Humans ,Electrical and Electronic Engineering ,Medical diagnosis ,Radiological and Ultrasound Technology ,medicine.diagnostic_test ,business.industry ,Representation (systemics) ,Pattern recognition ,medicine.disease ,Magnetic Resonance Imaging ,Computer Science Applications ,Identification (information) ,Autism spectrum disorder ,Biomarker (medicine) ,Artificial intelligence ,business ,Functional magnetic resonance imaging ,Algorithms ,Software - Abstract
Autism spectrum disorder (ASD) is a neurodevelopmental disorder that is characterized by a wide range of symptoms. Identifying biomarkers for accurate diagnosis is crucial for early intervention of ASD. While multi-site data increase sample size and statistical power, they suffer from inter-site heterogeneity. To address this issue, we propose a multi-site adaption framework via low-rank representation decomposition (maLRR) for ASD identification based on functional MRI (fMRI). The main idea is to determine a common low-rank representation for data from the multiple sites, aiming to reduce differences in data distributions. Treating one site as a target domain and the remaining sites as source domains, data from these domains are transformed ( i.e. , adapted) to a common space using low-rank representation. To reduce data heterogeneity between the target and source domains, data from the source domains are linearly represented in the common space by those from the target domain. We evaluated the proposed method on both synthetic and real multi-site fMRI data for ASD identification. The results suggest that our method yields superior performance over several state-of-the-art domain adaptation methods.
- Published
- 2020
40. Multi-Atlas Brain Parcellation Using Squeeze-and-Excitation Fully Convolutional Networks
- Author
-
Xianli Liu, Yang Li, Pew Thian Yap, Dinggang Shen, and Zhenyu Tang
- Subjects
Artificial neural network ,business.industry ,Computer science ,Deep learning ,Multi atlas ,Brain parcellation ,Pattern recognition ,Artificial intelligence ,business ,Computer Graphics and Computer-Aided Design ,Convolutional neural network ,Software - Abstract
Multi-atlas parcellation (MAP) is carried out on a brain image by propagating and fusing labelled regions from brain atlases. Typical nonlinear registration-based label propagation is time-consuming and sensitive to inter-subject differences. Recently, deep learning parcellation (DLP) has been proposed to avoid nonlinear registration for better efficiency and robustness than MAP. However, most existing DLP methods neglect using brain atlases, which contain high-level information (e.g., manually labelled brain regions), to provide auxiliary features for improving the parcellation accuracy. In this paper, we propose a novel multi-atlas DLP method for brain parcellation. Our method is based on fully convolutional networks (FCN) and squeeze-and-excitation (SE) modules. It can automatically and adaptively select features from the most relevant brain atlases to guide parcellation. Moreover, our method is trained via a generative adversarial network (GAN), where a convolutional neural network (CNN) with multi-scale $l_{1}$ loss is used as the discriminator. Benefiting from brain atlases, our method outperforms MAP and state-of-the-art DLP methods on two public image datasets (LPBA40 and NIREP-NA0).
- Published
- 2020
41. Leveraging Coupled Interaction for Multimodal Alzheimer’s Disease Diagnosis
- Author
-
Dinggang Shen, Yinghuan Shi, Seong-Whan Lee, Heung-Il Suk, and Yang Gao
- Subjects
Boosting (machine learning) ,Computer Networks and Communications ,Computer science ,Population ,Feature extraction ,Neuroimaging ,02 engineering and technology ,Machine Learning ,Alzheimer Disease ,Artificial Intelligence ,0202 electrical engineering, electronic engineering, information engineering ,medicine ,Humans ,Cognitive Dysfunction ,Diagnosis, Computer-Assisted ,education ,education.field_of_study ,medicine.diagnostic_test ,Artificial neural network ,business.industry ,Neurodegeneration ,Brain ,Reproducibility of Results ,Magnetic resonance imaging ,Pattern recognition ,Human brain ,medicine.disease ,Magnetic Resonance Imaging ,Computer Science Applications ,medicine.anatomical_structure ,Positron-Emission Tomography ,020201 artificial intelligence & image processing ,Neural Networks, Computer ,Artificial intelligence ,business ,Algorithms ,Software - Abstract
As the population becomes older worldwide, accurate computer-aided diagnosis for Alzheimer's disease (AD) in the early stage has been regarded as a crucial step for neurodegeneration care in recent years. Since it extracts the low-level features from the neuroimaging data, previous methods regarded this computer-aided diagnosis as a classification problem that ignored latent featurewise relation. However, it is known that multiple brain regions in the human brain are anatomically and functionally interlinked according to the current neuroscience perspective. Thus, it is reasonable to assume that the extracted features from different brain regions are related to each other to some extent. Also, the complementary information between different neuroimaging modalities could benefit multimodal fusion. To this end, we consider leveraging the coupled interactions in the feature level and modality level for diagnosis in this paper. First, we propose capturing the feature-level coupled interaction using a coupled feature representation. Then, to model the modality-level coupled interaction, we present two novel methods: 1) the coupled boosting (CB) that models the correlation of pairwise coupled-diversity on both inconsistently and incorrectly classified samples between different modalities and 2) the coupled metric ensemble (CME) that learns an informative feature projection from different modalities by integrating the intrarelation and interrelation of training samples. We systematically evaluated our methods with the AD neuroimaging initiative data set. By comparison with the baseline learning-based methods and the state-of-the-art methods that are specially developed for AD/MCI (mild cognitive impairment) diagnosis, our methods achieved the best performance with accuracy of 95.0% and 80.7% (CB), 94.9% and 79.9% (CME) for AD/NC (normal control), and MCI/NC identification, respectively.
- Published
- 2020
42. Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation
- Author
-
Yue Zhao, Xinbo Gao, Dinggang Shen, Chenqiang Gao, Zhiming Cui, Deyu Meng, Yang Liu, Chunfeng Lian, and Lingming Zhang
- Subjects
FOS: Computer and information sciences ,Scanner ,Computer science ,Computer Vision and Pattern Recognition (cs.CV) ,Concatenation ,Computer Science - Computer Vision and Pattern Recognition ,Surgical planning ,Discriminative model ,Image Processing, Computer-Assisted ,FOS: Electrical engineering, electronic engineering, information engineering ,Humans ,Segmentation ,Electrical and Electronic Engineering ,Radiological and Ultrasound Technology ,business.industry ,Deep learning ,Image and Video Processing (eess.IV) ,Pattern recognition ,Image segmentation ,Electrical Engineering and Systems Science - Image and Video Processing ,Computer Science Applications ,Graph (abstract data type) ,Neural Networks, Computer ,Artificial intelligence ,business ,Tooth ,Software - Abstract
Precise segmentation of teeth from intra-oral scanner images is an essential task in computer-aided orthodontic surgical planning. The state-of-the-art deep learning-based methods often simply concatenate the raw geometric attributes (i.e., coordinates and normal vectors) of mesh cells to train a single-stream network for automatic intra-oral scanner image segmentation. However, since different raw attributes reveal completely different geometric information, the naive concatenation of different raw attributes at the (low-level) input stage may bring unnecessary confusion in describing and differentiating between mesh cells, thus hampering the learning of high-level geometric representations for the segmentation task. To address this issue, we design a two-stream graph convolutional network (i.e., TSGCN), which can effectively handle inter-view confusion between different raw attributes to more effectively fuse their complementary information and learn discriminative multi-view geometric representations. Specifically, our TSGCN adopts two input-specific graph-learning streams to extract complementary high-level geometric representations from coordinates and normal vectors, respectively. Then, these single-view representations are further fused by a self-attention module to adaptively balance the contributions of different views in learning more discriminative multi-view representations for accurate and fully automatic tooth segmentation. We have evaluated our TSGCN on a real-patient dataset of dental (mesh) models acquired by 3D intraoral scanners. Experimental results show that our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation. Github: https://github.com/ZhangLingMing1/TSGCNet., Comment: 11 pages, 6 figures. arXiv admin note: text overlap with arXiv:2012.13697
- Published
- 2022
- Full Text
- View/download PDF
43. Longitudinal Prediction of Infant Diffusion MRI Data via Graph Convolutional Adversarial Networks
- Author
-
Dinggang Shen, Geng Chen, Pew Thian Yap, Weili Lin, Jaeil Kim, and Yoonmi Hong
- Subjects
Computer science ,Residual ,Convolutional neural network ,Article ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,Adversarial system ,Child Development ,0302 clinical medicine ,Longitudinal prediction ,Image Interpretation, Computer-Assisted ,Humans ,Electrical and Electronic Engineering ,Approximation theory ,Radiological and Ultrasound Technology ,business.industry ,Infant, Newborn ,Brain ,Pattern recognition ,Missing data ,Computer Science Applications ,Diffusion Magnetic Resonance Imaging ,Graph (abstract data type) ,Neural Networks, Computer ,Artificial intelligence ,business ,Software ,Diffusion MRI - Abstract
Missing data is a common problem in longitudinal studies due to subject dropouts and failed scans. We present a graph-based convolutional neural network to predict missing diffusion MRI data. In particular, we consider the relationships between sampling points in the spatial domain and the diffusion wave-vector domain to construct a graph. We then use a graph convolutional network to learn the non-linear mapping from available data to missing data. Our method harnesses a multi-scale residual architecture with adversarial learning for prediction with greater accuracy and perceptual quality. Experimental results show that our method is accurate and robust in the longitudinal prediction of infant brain diffusion MRI data.
- Published
- 2019
44. Multi-task Dynamic Transformer Network for Concurrent Bone Segmentation and Large-Scale Landmark Localization with Dental CBCT
- Author
-
Deqiang Xiao, Hung-Ying Lin, Steve Guofang Shen, Li Wang, Chunfeng Lian, Tianshu Kuang, James J. Xia, Pew Thian Yap, Dinggang Shen, Fan Wang, Jaime Gateno, and Hannah H. Deng
- Subjects
Landmark ,Computer science ,business.industry ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Multi-task learning ,02 engineering and technology ,Article ,030218 nuclear medicine & medical imaging ,law.invention ,03 medical and health sciences ,0302 clinical medicine ,law ,0202 electrical engineering, electronic engineering, information engineering ,Embedding ,020201 artificial intelligence & image processing ,Computer vision ,Segmentation ,Artificial intelligence ,Transformer ,business ,Bone segmentation - Abstract
Accurate bone segmentation and anatomical landmark localization are essential tasks in computer-aided surgical simulation for patients with craniomaxillofacial (CMF) deformities. To leverage the complementarity between the two tasks, we propose an efficient end-to-end deep network, i.e., multi-task dynamic transformer network (DTNet), to concurrently segment CMF bones and localize large-scale landmarks in one-pass from large volumes of cone-beam computed tomography (CBCT) data. Our DTNet was evaluated quantitatively using CBCTs of patients with CMF deformities. The results demonstrated that our method outperforms the other state-of-the-art methods in both tasks of the bony segmentation and the landmark digitization. Our DTNet features three main technical contributions. First, a collaborative two-branch architecture is designed to efficiently capture both fine-grained image details and complete global context for high-resolution volume-to-volume prediction. Second, leveraging anatomical dependencies between landmarks, regionalized dynamic learners (RDLs) are designed in the concept of “learns to learn” to jointly regress large-scale 3D heatmaps of all landmarks under limited computational costs. Third, adaptive transformer modules (ATMs) are designed for the flexible learning of task-specific feature embedding from common feature bases.
- Published
- 2021
45. Automatic Localization of Landmarks in Craniomaxillofacial CBCT Images Using a Local Attention-Based Graph Convolution Network
- Author
-
Deqiang Xiao, James J. Xia, Hannah H. Deng, Chunfeng Lian, David M. Alfi, Yankun Lang, Jaime Gateno, Peng Yuan, Dinggang Shen, Steve Guofang Shen, and Pew Thian Yap
- Subjects
Reconstructive surgery ,medicine.medical_specialty ,Landmark ,medicine.diagnostic_test ,Computer science ,business.industry ,Deep learning ,Computed tomography ,Article ,030218 nuclear medicine & medical imaging ,Convolution ,Automatic localization ,03 medical and health sciences ,0302 clinical medicine ,medicine ,Graph (abstract data type) ,Computer vision ,Artificial intelligence ,business ,030217 neurology & neurosurgery - Abstract
Landmark localization is an important step in quantifying craniomaxillofacial (CMF) deformities and designing treatment plans of reconstructive surgery. However, due to the severity of deformities and defects (partially missing anatomy), it is difficult to automatically and accurately localize a large set of landmarks simultaneously. In this work, we propose two cascaded networks for digitizing 60 anatomical CMF landmarks in cone-beam computed tomography (CBCT) images. The first network is a U-Net that outputs heatmaps for landmark locations and landmark features extracted with a local attention mechanism. The second network is a graph convolution network that takes the features extracted by the first network as input and determines whether each landmark exists via binary classification. We evaluated our approach on 50 sets of CBCT scans of patients with CMF deformities and compared them with state-of-the-art methods. The results indicate that our approach can achieve an average detection error of 1.47 mm with a false positive rate of 19%, outperforming related methods.
- Published
- 2021
46. Fast and Accurate Craniomaxillofacial Landmark Detection via 3D Faster R-CNN
- Author
-
Hannah H. Deng, Hung-Ying Lin, Pew Thian Yap, Xiaoyang Chen, Jaime Gateno, Tianshu Kuang, James J. Xia, Deqiang Xiao, Chunfeng Lian, and Dinggang Shen
- Subjects
Landmark ,Radiological and Ultrasound Technology ,medicine.diagnostic_test ,business.industry ,Computer science ,Deep learning ,Volume (computing) ,Computed tomography ,Landmark point ,Cone-Beam Computed Tomography ,Object detection ,Article ,Computer Science Applications ,Clinical Practice ,Imaging, Three-Dimensional ,medicine ,Computer vision ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,Software - Abstract
Automatic craniomaxillofacial (CMF) landmark localization from cone-beam computed tomography (CBCT) images is challenging, considering that 1) the number of landmarks in the images may change due to varying deformities and traumatic defects, and 2) the CBCT images used in clinical practice are typically large. In this paper, we propose a two-stage, coarse-to-fine deep learning method to tackle these challenges with both speed and accuracy in mind. Specifically, we first use a 3D faster R-CNN to roughly locate landmarks in down-sampled CBCT images that have varying numbers of landmarks. By converting the landmark point detection problem to a generic object detection problem, our 3D faster R-CNN is formulated to detect virtual, fixed-size objects in small boxes with centers indicating the approximate locations of the landmarks. Based on the rough landmark locations, we then crop 3D patches from the high-resolution images and send them to a multi-scale UNet for the regression of heatmaps, from which the refined landmark locations are finally derived. We evaluated the proposed approach by detecting up to 18 landmarks on a real clinical dataset of CMF CBCT images with various conditions. Experiments show that our approach achieves state-of-the-art accuracy of 0.89 ± 0.64mm in an average time of 26.2 seconds per volume.
- Published
- 2021
47. Future Trends of PET/MR and Utility of AI in Multi-Modal Imaging
- Author
-
Pew-Thian Yap, Mauricio Castillo, Mingxia Liu, Dinggang Shen, Weili Lin, and Sheng-Che Hung
- Subjects
Modal ,Computer science ,business.industry ,Computer vision ,Artificial intelligence ,business - Published
- 2021
48. Radiomics-based machine learning analysis and characterization of breast lesions with multiparametric diffusion-weighted MR
- Author
-
Dinggang Shen, Fuhua Yan, Zhicheng Jiao, Hong Zhu, Jie-Zhi Cheng, Xu Yan, Kun Sun, Weimin Chai, and Caixia Fu
- Subjects
Percentile ,Diffusion-weighted MRI ,Machine learning ,computer.software_genre ,General Biochemistry, Genetics and Molecular Biology ,Machine Learning ,Breast cancer ,Humans ,Medicine ,Breast ,Retrospective Studies ,Receiver operating characteristic ,business.industry ,Research ,General Medicine ,medicine.disease ,Random forest ,Support vector machine ,Diffusion Magnetic Resonance Imaging ,ROC Curve ,Feature (computer vision) ,Area Under Curve ,Principal component analysis ,Artificial intelligence ,business ,computer ,Diffusion MRI - Abstract
Background This study aimed to evaluate the utility of radiomics-based machine learning analysis with multiparametric DWI and to compare the diagnostic performance of radiomics features and mean diffusion metrics in the characterization of breast lesions. Methods This retrospective study included 542 lesions from February 2018 to November 2018. One hundred radiomics features were computed from mono-exponential (ME), biexponential (BE), stretched exponential (SE), and diffusion-kurtosis imaging (DKI). Radiomics-based analysis was performed by comparing four classifiers, including random forest (RF), principal component analysis (PCA), L1 regularization (L1R), and support vector machine (SVM). These four classifiers were trained on a training set with 271 patients via ten-fold cross-validation and tested on an independent testing set with 271 patients. The diagnostic performance of the mean diffusion metrics of ME (mADCall b, mADC0–1000), BE (mD, mD*, mf), SE (mDDC, mα), and DKI (mK, mD) were also calculated for comparison. The area under the receiver operating characteristic curve (AUC) was used to compare the diagnostic performance. Results RF attained higher AUCs than L1R, PCA and SVM. The AUCs of radiomics features for the differential diagnosis of breast lesions ranged from 0.80 (BE_D*) to 0.85 (BE_D). The AUCs of the mean diffusion metrics ranged from 0.54 (BE_mf) to 0.79 (ME_mADC0–1000). There were significant differences in the AUCs between the mean values of all diffusion metrics and radiomics features of AUCs (all P Conclusions The radiomics-based analysis of multiparametric DWI by RF enables better differentiation of benign and malignant breast lesions than the mean diffusion metrics.
- Published
- 2021
49. Learning MRI Artifact Removal With Unpaired Data
- Author
-
Siyuan Liu, Dinggang Shen, Pew Thian Yap, Liangqiong Qu, Kim-Han Thung, and Weili Lin
- Subjects
FOS: Computer and information sciences ,0301 basic medicine ,Matching (statistics) ,Computer Science - Machine Learning ,Computer Networks and Communications ,Computer science ,Image quality ,Computer Vision and Pattern Recognition (cs.CV) ,Computer Science - Computer Vision and Pattern Recognition ,Machine Learning (cs.LG) ,Image (mathematics) ,03 medical and health sciences ,0302 clinical medicine ,Artificial Intelligence ,FOS: Electrical engineering, electronic engineering, information engineering ,Unpaired Data ,Artifact (error) ,Artificial neural network ,business.industry ,Image and Video Processing (eess.IV) ,Supervised learning ,Pattern recognition ,Usability ,Electrical Engineering and Systems Science - Image and Video Processing ,Human-Computer Interaction ,030104 developmental biology ,Computer Vision and Pattern Recognition ,Artificial intelligence ,business ,030217 neurology & neurosurgery ,Software - Abstract
Retrospective artifact correction (RAC) improves image quality post acquisition and enhances image usability. Recent machine learning driven techniques for RAC are predominantly based on supervised learning and therefore practical utility can be limited as data with paired artifact-free and artifact-corrupted images are typically insufficient or even non-existent. Here we show that unwanted image artifacts can be disentangled and removed from an image via an RAC neural network learned with unpaired data. This implies that our method does not require matching artifact-corrupted data to be either collected via acquisition or generated via simulation. Experimental results demonstrate that our method is remarkably effective in removing artifacts and retaining anatomical details in images with different contrasts.
- Published
- 2021
50. Weakly Supervised Segmentation of COVID19 Infection with Scribble Annotation on CT Images
- Author
-
Kelei He, Xiaoming Liu, Yaozong Gao, Xiao Tang, Quan Yuan, Shuo Wang, Jinshan Tang, and Dinggang Shen
- Subjects
Pixel ,transformation consistency ,business.industry ,Computer science ,Deep learning ,infection segmentation ,COVID-19 ,Pattern recognition ,Convolutional neural network ,Article ,Image (mathematics) ,Task (project management) ,Annotation ,Transformation (function) ,Artificial Intelligence ,Signal Processing ,Segmentation ,Computer Vision and Pattern Recognition ,Artificial intelligence ,business ,uncertainty ,Software ,weakly supervised learning - Abstract
Segmentation of infections from CT scans is important for accurate diagnosis and follow-up in tackling the COVID-19. Although the convolutional neural network has great potential to automate the segmentation task, most existing deep learning-based infection segmentation methods require fully annotated ground-truth labels for training, which is time-consuming and labor-intensive. This paper proposed a novel weakly supervised segmentation method for COVID-19 infections in CT slices, which only requires scribble supervision and is enhanced with the uncertainty-aware self-ensembling and transformation-consistent techniques. Specifically, to deal with the difficulty caused by the shortage of supervision, an uncertainty-aware mean teacher is incorporated into the scribble-based segmentation method, encouraging the segmentation predictions to be consistent under different perturbations for an input image. This mean teacher model can guide the student model to be trained using information in images without requiring manual annotations. On the other hand, considering the output of the mean teacher contains both correct and unreliable predictions, equally treating each prediction in the teacher model may degrade the performance of the student network. To alleviate this problem, the pixel level uncertainty measure on the predictions of the teacher model is calculated, and then the student model is only guided by reliable predictions from the teacher model. To further regularize the network, a transformation-consistent strategy is also incorporated, which requires the prediction to follow the same transformation if a transform is performed on an input image of the network. The proposed method has been evaluated on two public datasets and one local dataset. The experimental results demonstrate that the proposed method is more effective than other weakly supervised methods and achieves similar performance as those fully supervised.
- Published
- 2021
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.