Search

Cortical surface registration is an essential step and prerequisite for surface-based neuroimaging analysis. It aligns cortical surfaces across individuals and time points to establish cross-sectional and longitudinal cortical correspondences to facilitate neuroimaging studies. Though achieving good performance, available methods are either time consuming or not flexible to extend to multiple or high dimensional features. Considering the explosive availability of large-scale and multimodal brain MRI data, fast surface registration methods that can flexibly handle multimodal features are desired. In this study, we develop a Superfast Spherical Surface Registration (S3Reg) framework for the cerebral cortex. Leveraging an end-to-end unsupervised learning strategy, S3Reg offers great flexibility in the choice of input feature sets and output similarity measures for registration, and meanwhile reduces the registration time significantly. Specifically, we exploit the powerful learning capability of spherical Convolutional Neural Network (CNN) to directly learn the deformation fields in spherical space and implement diffeomorphic design with “scaling and squaring” layers to guarantee topology-preserving deformations. To handle the polar-distortion issue, we construct a novel spherical CNN model using three orthogonal Spherical U-Nets. Experiments are performed on two different datasets to align both adult and infant multimodal cortical features. Results demonstrate that our S3Reg shows superior or comparable performance with state-of-the-art methods, while improving the registration time from 1 min to 10 sec.

Parkinson's disease (PD) is characterized by complex clinical symptoms, including classic motor and nonmotor disturbances. Patients with PD vary in clinical manifestations and prognosis, which point to the existence of subtypes. This study aimed to find the fiber connectivity correlations with several crucial clinical symptoms and identify PD subtypes using unsupervised clustering analysis. One hundred and thirty‐four PD patients and 77 normal controls were enrolled. Canonical correlation analysis (CCA) was performed to define the clinically relevant connectivity features, which were then used in the hierarchical clustering analysis to identify the distinct subtypes of PD patients. Multimodal neuroimaging analyses were further used to explore the neurophysiological basis of these subtypes. The methodology was validated in an independent data set. CCA revealed two significant clinically relevant patterns (motor‐related pattern and depression‐related pattern; r = .94, p, Parkinson's disease (PD) is a heterogeneous syndrome, which points to the subtypes in PD patients. This study used a data‐driven approach based on fiber connectivity to automatically identify subtypes in PD patients with distinct clinical features. Neuroimaging analyses revealed significant brain alterations behind each subtype.

Studying the early dynamic development of cortical folding with remarkable individual variability is critical for understanding normal early brain development and related neurodevelopmental disorders. This study focuses on the fingerprinting capability and the individual variability of cortical folding during early brain development. Specifically, we aim to explore (a) whether the developing neonatal cortical folding is unique enough to be considered as a "fingerprint" that can reliably identify an individual within a cohort of infants; (b) which cortical regions manifest more individual variability and thus contribute more for infant identification; (c) whether the infant twins can be distinguished by cortical folding. Hence, for the first time, we conduct infant individual identification and individual variability analysis involving twins based on the developing cortical folding features (mean curvature, average convexity, and sulcal depth) in 472 neonates with 1,141 longitudinal MRI scans. Experimental results show that the infant individual identification achieves 100% accuracy when using the neonatal cortical folding features to predict the identities of 1- and 2-year-olds. Besides, we observe high identification capability in the high-order association cortices (i.e., prefrontal, lateral temporal, and inferior parietal regions) and two unimodal cortices (i.e., precentral gyrus and lateral occipital cortex), which largely overlap with the regions encoding remarkable individual variability in cortical folding during the first 2 years. For twins study, we show that even for monozygotic twins with identical genes and similar developmental environments, their cortical folding features are unique enough for accurate individual identification; and in some high-order association cortices, the differences between monozygotic twin pairs are significantly lower than those between dizygotic twins. This study thus provides important insights into individual identification and individual variability based on cortical folding during infancy.

Biomedical image fusion is the process of combining the information from different imaging modalities to get a synthetic image. Fusion of phase contrast and green fluorescent protein (GFP) images is significant to predict the role of unknown proteins, analyze the function of proteins, locate the subcellular structure, and so forth. Generally, the fusion performance largely depends on the registration of GFP and phase contrast images. However, accurate registration of multi-modal images is a very challenging task. Hence, we propose a novel fusion method based on convolutional sparse representation (CSR) to fuse the mis-registered GFP and phase contrast images. At first, the GFP and phase contrast images are decomposed by CSR to get the coefficients of base layers and detail layers. Secondly, the coefficients of detail layers are fused by the sum modified Laplacian (SML) rule while the coefficients of base layers are fused by the proposed adaptive region energy (ARE) rule. ARE rule is calculated by discussion mechanism based brain storm optimization (DMBSO) algorithm. Finally, the fused image is achieved by carrying out the inverse CSR. The proposed fusion method is tested on 100 pairs of mis-registered GFP and phase contrast images. The experimental results reveal that our proposed fusion method exhibits better fusion results and superior robustness than several existing fusion methods.

Media-adventitia (MA) border delineates the outer appearance of arterial wall in intravascular ultrasound (IVUS) image. The detection of MA border is a challenging topic due to many difficulties such as complicated intravascular structures, intrinsic artifacts and image noises. We propose a classification-based MA border detection method with an embedded feature selection technique. The feature selection technique is based on Fractional-order Darwinian particle swarm optimization (FODPSO) algorithm. By employing feature selection, 293-dimension features including multi-scale features, gray-scale features and morphological feature are reducing to 37-dimension. The border detection method with feature selection is tested on a public dataset extracted from in-vivo pullbacks of human coronary arteries, which contains 77 IVUS images. Three indicators, Jaccard (JACC), Hausdorff Distance (HD) and Percentage of Area Difference (PAD), are measured for quantitative evaluation. Detection with 293-dimension features obtains JACC 0.79, HD 1.41 and PAD 0.16, while detection with 37-dimension features obtains JACC 0.83, HD 1.27 and PAD 0.12, indicating that the FODPSO-based feature selection method improves MA border detection by JACC 0.04, HD 0.14 and PAD 0.04. Furthermore, the proposed border detection method acquires better performances compared with two other automatic methods conducted on the same dataset available in literature.

Current spherical surface registration methods achieve good performance on alignment and spatial normalization of cortical surfaces across individuals in neuroimaging analysis. However, they are computationally intensive, since they have to optimize an objective function independently for each pair of surfaces. In this paper, we present a fast learning-based algorithm that makes use of the recent development in spherical Convolutional Neural Networks (CNNs) for spherical cortical surface registration. Given a set of surface pairs without supervised information such as ground truth deformation fields or anatomical landmarks, we formulate the registration as a parametric function and learn its parameters by enforcing the feature similarity between one surface and the other one warped by the estimated deformation field using the function. Then, given a new pair of surfaces, we can quickly infer the spherical deformation field registering one surface to the other one. We model this parametric function using three orthogonal Spherical U-Nets and use spherical transform layers to warp the spherical surfaces, while imposing smoothness constraints on the deformation field. All the layers in the network are well-defined and differentiable, thus the parameters can be effectively learned. We show that our method achieves accurate cortical alignment results on 102 subjects, comparable to two state-of-the-art methods: Spherical Demons and MSM, while runs much faster.

Convolutional Neural Networks (CNNs) have achieved overwhelming success in learning-related problems for 2D/3D images in the Euclidean space. However, unlike in the Euclidean space, the shapes of many structures in medical imaging have an inherent spherical topology in a manifold space, e.g., the convoluted brain cortical surfaces represented by triangular meshes. There is no consistent neighborhood definition and thus no straightforward convolution/pooling operations for such cortical surface data. In this paper, leveraging the regular and hierarchical geometric structure of the resampled spherical cortical surfaces, we create the 1-ring filter on spherical cortical triangular meshes and accordingly develop convolution/pooling operations for constructing Spherical U-Net for cortical surface data. However, the regular nature of the 1-ring filter makes it inherently limited to model fixed geometric transformations. To further enhance the transformation modeling capability of Spherical U-Net, we introduce the deformable convolution and deformable pooling to cortical surface data and accordingly propose the Spherical Deformable U-Net (SDU-Net). Specifically, spherical offsets are learned to freely deform the 1-ring filter on the sphere to adaptively localize cortical structures with different sizes and shapes. We then apply the SDU-Net to two challenging and scientifically important tasks in neuroimaging: cortical surface parcellation and cortical attribute map prediction. Both applications validate the competitive performance of our approach in accuracy and computational efficiency in comparison with state-of-the-art methods.

Cortical surface registration and parcellation are two essential steps in neuroimaging analysis. Conventionally, they are performed independently as two tasks, ignoring the inherent connections of these two closely-related tasks. Essentially, both tasks rely on meaningful cortical feature representations, so they can be jointly optimized by learning shared useful cortical features. To this end, we propose a deep learning framework for joint cortical surface registration and parcellation. Specifically, our approach leverages the spherical topology of cortical surfaces and uses a spherical network as the shared encoder to first learn shared features for both tasks. Then we train two task-specific decoders for registration and parcellation, respectively. We further exploit the more explicit connection between them by incorporating the novel parcellation map similarity loss to enforce the boundary consistency of regions, thereby providing extra supervision for the registration task. Conversely, parcellation network training also benefits from the registration, which provides a large amount of augmented data by warping one surface with manual parcellation map to another surface, especially when only few manually-labeled surfaces are available. Experiments on a dataset with more than 600 cortical surfaces show that our approach achieves large improvements on both parcellation and registration accuracy (over separately trained networks) and enables training high-quality parcellation and registration models using much fewer labeled data.

Spatiotemporal (4D) cortical surface atlas during infancy plays an important role for surface-based visualization, normalization and analysis of the dynamic early brain development. Conventional atlas construction methods typically rely on classical group-wise registration on sub-populations and ignore longitudinal constraints, thus having three main issues: 1) constructing templates at discrete time points; 2) resulting in longitudinal inconsistency among different age’s atlases; and 3) taking extremely long runtime. To address these issues, in this paper, we propose a fast unsupervised learning-based surface atlas construction framework incorporating longitudinal constraints to enforce the within-subject temporal correspondence in the atlas space. To well handle the difficulties of learning large deformations, we propose a multi-level multi-modal spherical registration network to perform cortical surface registration in a coarse-to-fine manner. Thus, only small deformations need to be estimated at each resolution level using the registration network, which further improves registration accuracy and atlas quality. Our constructed 4D infant cortical surface atlas based on 625 longitudinal scans from 291 infants is temporally continuous, in contrast to the state-of-the-art UNC 4D Infant Surface Atlas, which only provides the atlases at a few discrete sparse time points. By evaluating the intra- and inter-subject spatial normalization accuracy after alignment onto the atlas, our atlas demonstrates more detailed and fine-grained cortical patterns, thus leading to higher accuracy in surface registration.

Purpose: To investigate the structural covariance network disruption in Parkinson's disease (PD), and explore the functional alterations of disrupted structural covariance network. Methods: A cohort of 100 PD patients and 70 healthy participants underwent structural and functional magnetic resonance scanning. Independent component analysis (ICA) was applied separately to both deformation-based morphometry (DBM) maps and functional maps with the same calculating parameters (both decomposed into 20 independent components (ICs) and computed 20 times the Infomax algorithm in ICASSO). Disrupted structural covariance network in PD patients was identified, and then, we performed goodness of fit analysis to obtain the functional network that showed the highest spatial overlap with it. We investigated the relationship between structural covariance network and functional network alterations. Finally, to further understand the structural and functional alterations over time, we performed a longitudinal subgroup analysis (51 patients were followed up for 2 years) with the same procedures. Results: In a cross-sectional analysis, PD patients showed decreased structural covariance between anterior and posterior cingulate subnetworks. The functional components showed best overlap with anterior and posterior cingulate structural subnetworks were selected as anterior and posterior cingulate functional subnetworks. The functional connectivity between them was significantly increased [assessed by Functional Network Connectivity (FNC) toolbox]; and the increased functional connectivity was negatively correlated with cingulate structural covariance network integrity. Longitudinal subgroup analysis showed cingulate structural covariance network disruption was worse at follow-up, while the functional connectivity between anterior and posterior cingulate network was increased at baseline and decreased at follow-up. Conclusion: This study indicated that the cingulate structural covariance network displayed a high susceptibility in PD patients. This study indicated that the cingulate structural covariance network displayed a high susceptibility in PD patients. Considering that disrupted structural covariance network coexisted with enhanced/remained functional activity during disease development, enhanced functional activity underlying the disrupted cingulate structural covariance network might represent a temporal compensation for maintaining clinical performance.

The detection of the media-adventitia (MA) border in intravascular ultrasound (IVUS) images is essential for vessel assessment and disease diagnosis. However, it remains a challenging task, considering the existence of plaque, calcification, and various artifacts. In this article, an effective method based on classification is proposed to extract the MA border in IVUS images. First, a novel morphologic feature describing the relative position of each structure relative to the MA border, called RPES for short, is proposed. Then, the RPES feature and other features are employed in a multiclass extreme learning machine (ELM) to classify IVUS images into nine classes including the MA border and other structures. At last, a modified snake model is employed to effectively detect the MA border in the rectangular domain, in which a modified external force field is constructed on the basis of local border appearances and classification results. The proposed method is evaluated on a public dataset with 77 IVUS images by three indicators in eight situations, such as calcification and a guide wire artifact. With the proposed RPES feature, detection performances are improved by more than 39 percent, which shows an apparent advantage in comparative experiments. Furthermore, compared with two other existing methods used on the same dataset, the proposed method achieves 18 of the best indicators among 24, demonstrating its higher capability in detecting the MA border.

Image fusion techniques can integrate the information from different imaging modalities to get a composite image which is more suitable for human visual perception and further image processing tasks. Fusing green fluorescent protein (GFP) and phase contrast images is very important for subcellular localization, functional analysis of protein and genome expression. The fusion method of GFP and phase contrast images based on complex shearlet transform (CST) is proposed in this paper. Firstly the GFP image is converted to IHS model and its intensity component is obtained. Secondly the CST is performed on the intensity component and the phase contrast image to acquire the low-frequency subbands and the high-frequency subbands. Then the high-frequency subbands are merged by the absolute-maximum rule while the low-frequency subbands are merged by the proposed Haar wavelet-based energy (HWE) rule. Finally the fused image is obtained by performing the inverse CST on the merged subbands and conducting IHS-to-RGB conversion. The proposed fusion method is tested on a number of GFP and phase contrast images and compared with several popular image fusion methods. The experimental results demonstrate that the proposed fusion method can provide better fusion results in terms of subjective quality and objective evaluation.

In many applications, image deblurring is a pre-requisite to improve the sharpness of an image before it can be further processed. Iterative methods are widely used for deblurring images but care must be taken to ensure that the iterative process is robust, meaning that the process does not diverge and reaches the solution reasonably fast, two goals that sometimes compete against each other. In practice, it remains challenging to choose parameters for the iterative process to be robust. We propose a new approach consisting of relaxed initialization and pixel-wise updates of the step size for iterative methods to achieve robustness. The first novel design of the approach is to modify the initialization of existing iterative methods to stop a noise term from being propagated throughout the iterative process. The second novel design is the introduction of a vectorized step size that is adaptively determined through the iteration to achieve higher stability and accuracy in the whole iterative process. The vectorized step size aims to update each pixel of an image individually, instead of updating all the pixels by the same factor. In this work, we implemented the above designs based on the Landweber method to test and demonstrate the new approach. Test results showed that the new approach can deblur images from noisy observations and achieve a low mean squared error with a more robust performance.

Cortical folding of the adult brain is highly convoluted and encodes inter-subject variable characteristics. Recent studies suggest that it is useful for individual identification in adults. However, little is known about whether the infant cortical folding, which undergoes dynamic postnatal development, can be used for individual identification. To fill this gap, we propose to explore cortical folding patterns for infant subject identification. This study thus aims to address two important questions in neuroscience: 1) whether the infant cortical folding is unique for individual identification; and 2) considering the region-specific inter-subject variability, which cortical regions are more distinct and reliable for infant identification. To this end, we propose a novel discriminative descriptor of regional cortical folding based on multi-scale analysis of curvature maps via spherical wavelets, called FoldingPrint. Experiments are carried out on a large longitudinal dataset with 1,141 MRI scans from 472 infants. Despite the dramatic development in the first two years, successful identification of 1-year-olds and 2-year-olds using their neonatal cortical folding (with accuracy > 98%) indicates the effectiveness of the proposed method. Moreover, we reveal that regions with high identification accuracy and large inter-subject variability mainly distribute in high-order association cortices.

Increasing multi-site infant neuroimaging datasets are facilitating the research on understanding early brain development with larger sample size and bigger statistical power. However, a joint analysis of cortical properties (e.g., cortical thickness) is unavoidably facing the problem of non-biological variance introduced by differences in MRI scanners. To address this issue, in this paper, we propose cycle-consistent adversarial networks based on spherical cortical surface to harmonize cortical thickness maps between different scanners. We combine the spherical U-Net and CycleGAN to construct a surface-to-surface CycleGAN (S2SGAN). Specifically, we model the harmonization from scanner X to scanner Y as a surface-to-surface translation task. The first goal of harmonization is to learn a mapping \(G_X: X\rightarrow Y\) such that the distribution of surface thickness maps from \(G_X(X)\) is indistinguishable from Y. Since this mapping is highly under-constrained, with the second goal of harmonization to preserve individual differences, we utilize the inverse mapping \(G_Y: Y\rightarrow X\) and the cycle consistency loss to enforce \(G_Y(G_X(X))\approx X\) (and vice versa). Furthermore, we incorporate the correlation coefficient loss to guarantee the structure consistency between the original and the generated surface thickness maps. Quantitative evaluation on both synthesized and real infant cortical data demonstrates the superior ability of our method in removing unwanted scanner effects and preserving individual differences simultaneously, compared to the state-of-the-art methods.

International audience; Surgical tool detection is attracting increasing attention from the medical image analysis community. The goal generally is not to precisely locate tools in images, but rather to indicate which tools are being used by the surgeon at each instant. The main motivation for annotating tool usage is to design efficient solutions for surgical workflow analysis, with potential applications in report generation, surgical training and even real-time decision support. Most existing tool annotation algorithms focus on laparo-scopic surgeries. However, with 19 million interventions per year, the most common surgical procedure in the world is cataract surgery. The CATARACTS challenge was organized in 2017 to evaluate tool annotation algorithms in the specific context of cataract surgery. It relies on more than nine hours of videos, from 50 cataract surgeries, in which the presence of 21 surgical tools was manually annotated by two experts. With 14 participating teams, this challenge can be considered a success. As might be expected,-H. Conze et al. / Medical Image Analysis 52 (2019) 24-41 25 the submitted solutions are based on deep learning. This paper thoroughly evaluates these solutions: in particular, the quality of their annotations are compared to that of human interpretations. Next, lessons learnt from the differential analysis of these solutions are discussed. We expect that they will guide the design of efficient surgery monitoring tools in the near future.

Convolutional Neural Networks (CNNs) have been providing the state-of-the-art performance for learning-related problems involving 2D/3D images in Euclidean space. However, unlike in the Euclidean space, the shapes of many structures in medical imaging have a spherical topology in a manifold space, e.g., brain cortical or subcortical surfaces represented by triangular meshes, with large inter-subject and intra-subject variations in vertex number and local connectivity. Hence, there is no consistent neighborhood definition and thus no straightforward convolution/transposed convolution operations for cortical/subcortical surface data. In this paper, by leveraging the regular and consistent geometric structure of the resampled cortical surface mapped onto the spherical space, we propose a novel convolution filter analogous to the standard convolution on the image grid. Accordingly, we develop corresponding operations for convolution, pooling, and transposed convolution for spherical surface data and thus construct spherical CNNs. Specifically, we propose the Spherical U-Net architecture by replacing all operations in the standard U-Net with their spherical operation counterparts. We then apply the Spherical U-Net to two challenging and neuroscientifically important tasks in infant brains: cortical surface parcellation and cortical attribute map development prediction. Both applications demonstrate the competitive performance in the accuracy, computational efficiency, and effectiveness of our proposed Spherical U-Net, in comparison with the state-of-the-art methods.

A method based on intelligent video processing is introduced here to objectively assess the intensity and energy expenditure of popular indoor workouts, including sit up, push up, jumping jack, and squat. The method is based a smart phone camera, which can count the repetition, assess the intensity level, and quantify the energy expenditure of the workouts. It uses an algorithm that analyzes body movement and body’s effort to overcome gravity in hierarchical layers, each with an increasing level of details. The energy expenditure values for different types of workouts at different intensities determined by the method are compared with a gold standard indirect calorimetry apparatus worn by the study subject.

Background Patients with the postural instability and gait difficulty subtype (PIGD) of Parkinson's disease (PD) are a refractory challenge in clinical practice. Despite previous attempts that have been made at studying subtype-specific brain alterations across PD population, conclusive neuroimaging biomarkers on patients with the PIGD subtype are still lacking. Machine learning-based classifications are a promising tool for differential diagnosis that effectively integrate complex and multivariate data. Objective Our present study aimed to introduce the machine learning-based automatic classification for the first time to distinguish patients with the PIGD subtype from those with the non-PIGD subtype of PD at the individual level. Methods Fifty-two PD patients and forty-five normal controls (NCs) were recruited and underwent multi-modal MRI scans including a set of resting-state functional, 3D T1-weighted and diffusion tensor imaging sequences. By comparing the PD patients with the NCs, features that were not conducive to the subtype-specific classification were ruled out from massive brain features. We applied a support vector machine classifier with the recursive feature elimination method to multi-modal MRI data for selecting features with the best discriminating power, and evaluated the proposed classifier with the leave-one-out cross-validation. Results Using this classifier, we obtained satisfactory diagnostic rates (accuracy = 92.31%, specificity = 96.97%, sensitivity = 84.21% and AUCmax = 0.9585). The diagnostic agreement evaluated by the Kappa test showed an almost perfect agreement with the existing clinical categorization (Kappa value = 0.83). Conclusions With these favorable results, our findings suggested the machine learning-based classification as an alternative technique to classifying clinical subtypes in PD.

In this paper, we presented a new parameter-free thresholding method for image segmentation. In separating an image into two classes, the method employs an objective function that not only maximizes the between-class variance but also the distance between the mean of each class and the global mean of the image. The design of the objective function aims to circumvent the challenge that many existing techniques encounter when the underlying two classes have very different sizes or variances. The advantages of the new method are twofold. First, it is parameter-free, meaning that it can generate consistent results. Second, the new method has a simple form that makes it easy to adapt to different applications. We tested and compared the new method with the standard Otsu method, the maximum entropy method, and the 2D Otsu method on the simulated and real biomedical and photographic images and found that the new method can achieve a more accurate and robust performance.

The paper presents a novel approach for feature selection based on extreme learning machine (ELM) and Fractional-order Darwinian particle swarm optimization (FODPSO) for regression problems. The proposed method constructs a fitness function by calculating mean square error (MSE) acquired from ELM. And the optimal solution of the fitness function is searched by an improved particle swarm optimization, FODPSO. In order to evaluate the performance of the proposed method, comparative experiments with other relative methods are conducted in seven public datasets. The proposed method obtains six lowest MSE values among all the comparative methods. Experimental results demonstrate that the proposed method has the superiority of getting lower MSE with the same scale of feature subset or requiring smaller scale of feature subset for similar MSE.