Search

The papers in this special section focus on generative adversarial networks in biomedical image computing. The field of biomedical imaging has obtained great progress from Roentgen’s original discovery of the X-ray to the current imaging tools, including Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), Computed Tomography (CT), and Ultrasound (US). The benefits of using these non-invasive imaging technologies are to assess the current condition of an organ or tissue, which can be used to monitor a patient over time over time for accurate and timely diagnosis and treatment.With the development of imaging technologies, developing advanced artificial intelligence algorithms for automated image analysis has shown the potential to change many aspects of clinical applications within the next decade. Meanwhile, these advanced technologies have also brought new issues and challenges. Thus, there has been a growing demand for biomedical imaging computing to be a component of clinical trials and device improvement. Currently, Generative adversarial networks (GANs) have been attached growing interests in the computer vision community due to their capability of data generation or translation. GAN-based models are able to learn from a set of training data and generate new data with the same characteristics as the training ones, which have also proven to be the state of the art for generating sharp and realistic images. More importantly, GAN has been rapidly applied to many traditional and novel applications in the medical domain, such as image reconstruction, segmentation, diagnosis, synthesis, and so on. Despite GAN substantial progress in these areas, their application to medical image computing still faces challenges and unsolved problems remain.

This paper describes an approach based on machine learning technology that is of particular interest for the localization and characterization of both single focal stenoses and multivessel multifocal lesions. Due to the complexity of analyzing large amounts of data for the cardiac surgeon, we pay special attention to the analysis, training, and comparison of popular neural networks that classify and localize foci of stenosis on coronary angiography data. From the complete coronarography dataset collected at the Research Institute for Complex Issues of Cardiovascular Diseases, we retrospectively select data of 100 patients. For the automated analysis of the medical data, the paper considers in detail three models (SSD MobileNet V1, Faster-RCNN ResNet-50 V1, and Faster-RCNN NASNet), which differ in their architecture, complexity, and the number of weights. The models are compared in terms of their basic efficiency characteristics: accuracy, training time, and prediction time. The test results show that the training and prediction times are directly proportional to the complexity of the models. In this regard, Faster-RCNN NASNet exhibits the lowest prediction time (the average processing time for one image is 880 ms), while Faster-RCNN ResNet-50 V1 has the highest prediction accuracy. The latter model reaches the mean average precision (mAP) level of 0.92 on the validation dataset. On the other hand, SSD MobileNet V1 is the fastest model, capable of making predictions with a prediction rate of 23 fps.

Despite its success in the field of minimally invasive surgery, endoscopy image analysis remains challenging due to limited image settings and control conditions. The low resolution and existence of large number of reflections in endoscopy images are the major problems in the automatic detection of objects. To address these issues, we presented a novel framework based on the convolutional neural networks. The proposed approach consists of three major parts. First, a deep learning (DL)-based image evaluation method is used to classify the input images into two groups, namely, specular highlights and weakly illuminated groups. Second, the specular highlight is detected using the DL-based method, and the reflected areas are recovered through a patch-based restoration operation. Lastly, gamma correction with optimized reflectance and illumination estimation is adopted to enhance the weakly illuminated images. The proposed method is compared against the existing ones, and the experimental results demonstrate that the former outperforms the latter in terms of subjective and objective assessments. This finding indicates that the proposed approach can serve as a potential tool for improving the quality of the endoscopy images used to examine internal body organs.

Machine learning methods have been widely used for early diagnosis of Alzheimer's disease (AD) via functional connectivity networks (FCNs) analysis from neuroimaging data. The conventional low-order FCNs are obtained by time-series correlation of the whole brain based on resting-state functional magnetic resonance imaging (R-fMRI). However, FCNs overlook inter-region interactions, which limits application to brain disease diagnosis. To overcome this drawback, we develop a novel framework to exploit the high-level dynamic interactions among brain regions for early AD diagnosis. Specifically, a sliding window approach is employed to generate some R-fMRI sub-series. The correlations among these sub-series are then used to construct a series of dynamic FCNs. High-order FCNs based on the topographical similarity between each pair of the dynamic FCNs are then constructed. Afterward, a local weight clustering method is used to extract effective features of the network, and the least absolute shrinkage and selection operation method is chosen for feature selection. A support vector machine is employed for classification, and the dynamic high-order network approach is evaluated on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Our experimental results demonstrate that the proposed approach not only achieves promising results for AD classification, but also successfully recognizes disease-related biomarkers.

Accurate segmentation of pediatric echocardiography images is essential for a wide range of diagnostic and pre-interventional planning, but remains challenging (e.g., low signal to noise ratio and internal variability in heart appearance). To address these problems, in this paper, we propose a novel Cardiac Attention-guided Dual-path Network (i.e., AIDAN). AIDAN comprises a convolutional block attention module (CBAM) attached to a spatial (i.e., SPA) and context paths (i.e., CPA), which can guide the network and learn the most discriminative features. The spatial path captures low-level spatial features, and the context path is designed to exploit high-level context. Finally, features learned from the two paths are fused efficiently using a specially designed feature fusion module (FFM), and these are used to predict the final segmentation map. We experiment on a self-collected dataset of 127 pediatric echocardiography cases which are videos containing at least a complete cardiac cycle, and obtain a Dice coefficient of 0.951 and 0.914, in the left ventricle and atrium segments, respectively. AIDAN outperforms other state-of-the-art methods and has great potential for pediatric echocardiography images analysis.

Invasive coronary angiography remains the gold standard for diagnosing coronary artery disease, which may be complicated by both, patient-specific anatomy and image quality. Deep learning techniques aimed at detecting coronary artery stenoses may facilitate the diagnosis. However, previous studies have failed to achieve superior accuracy and performance for real-time labeling. Our study is aimed at confirming the feasibility of real-time coronary artery stenosis detection using deep learning methods. To reach this goal we trained and tested eight promising detectors based on different neural network architectures (MobileNet, ResNet-50, ResNet-101, Inception ResNet, NASNet) using clinical angiography data of 100 patients. Three neural networks have demonstrated superior results. The network based on Faster-RCNN Inception ResNet V2 is the most accurate and it achieved the mean Average Precision of 0.95, F1-score 0.96 and the slowest prediction rate of 3 fps on the validation subset. The relatively lightweight SSD MobileNet V2 network proved itself as the fastest one with a low mAP of 0.83, F1-score of 0.80 and a mean prediction rate of 38 fps. The model based on RFCN ResNet-101 V2 has demonstrated an optimal accuracy-to-speed ratio. Its mAP makes up 0.94, F1-score 0.96 while the prediction speed is 10 fps. The resultant performance-accuracy balance of the modern neural networks has confirmed the feasibility of real-time coronary artery stenosis detection supporting the decision-making process of the Heart Team interpreting coronary angiography findings. ispartof: SCIENTIFIC REPORTS vol:11 issue:1 ispartof: location:England status: published

Shape reconstruction from sparse point clouds/images is a challenging and relevant task required for a variety of applications in computer vision and medical image analysis (e.g. surgical navigation, cardiac motion analysis, augmented/virtual reality systems). A subset of such methods, viz. 3D shape reconstruction from 2D contours, is especially relevant for computer-aided diagnosis and intervention applications involving meshes derived from multiple 2D image slices, views or projections. We propose a deep learning architecture, coined Mesh Reconstruction Network (MR-Net), which tackles this problem. MR-Net enables accurate 3D mesh reconstruction in real-time despite missing data and with sparse annotations. Using 3D cardiac shape reconstruction from 2D contours defined on short-axis cardiac magnetic resonance image slices as an exemplar, we demonstrate that our approach consistently outperforms state-of-the-art techniques for shape reconstruction from unstructured point clouds. Our approach can reconstruct 3D cardiac meshes to within 2.5-mm point-to-point error, concerning the ground-truth data (the original image spatial resolution is ∼1.8×1.8×10mm3). We further evaluate the robustness of the proposed approach to incomplete data, and contours estimated using an automatic segmentation algorithm. MR-Net is generic and could reconstruct shapes of other organs, making it compelling as a tool for various applications in medical image analysis. ispartof: MEDICAL IMAGE ANALYSIS vol:74 ispartof: location:Netherlands status: published

A new multimodal biometric database designed and acquired within the framework of the European BioSecure Network of Excellence is presented. It is comprised of more than 600 individuals acquired simultaneously in three scenarios: 1) over the Internet, 2) in an office environment with desktop PC, and 3) in indoor/outdoor environments with mobile portable hardware. The three scenarios include a common part of audio/video data. Also, signature and fingerprint data have been acquired both with desktop PC and mobile portable hardware. Additionally, hand and iris data were acquired in the second scenario using desktop PC. Acquisition has been conducted by 11 European institutions. Additional features of the BioSecure Multimodal Database (BMDB) are: two acquisition sessions, several sensors in certain modalities, balanced gender and age distributions, multimodal realistic scenarios with simple and quick tasks per modality, cross-European diversity, availability of demographic data, and compatibility with other multimodal databases. The novel acquisition conditions of the BMDB allow us to perform new challenging research and evaluation of either monomodal or multimodal biometric systems, as in the recent BioSecure Multimodal Evaluation campaign. A description of this campaign including baseline results of individual modalities from the new database is also given. The database is expected to be available for research purposes through the BioSecure Association during 2008, Published at IEEE Transactions on Pattern Analysis and Machine Intelligence journal

Munro's Microabscess (MM) is the diagnostic hallmark of psoriasis. Neutrophil detection in the Stratum Corneum (SC) of the skin epidermis is an integral part of MM detection in skin biopsy. The microscopic inspection of skin biopsy is a tedious task and staining variations in skin histopathology often hinder human performance to differentiate neutrophils from skin keratinocytes. Motivated from this, we propose a computational framework that can assist human experts and reduce potential errors in diagnosis. The framework first segments the SC layer, and multiple patches are sampled from the segmented regions which are classified to detect neutrophils. Both UNet and CapsNet are used for segmentation and classification. Experiments show that of the two choices, CapsNet, owing to its robustness towards better hierarchical object representation and localisation ability, appears as a better candidate for both segmentation and classification tasks and hence, we termed our framework as MICaps. The training algorithm explores both minimisation of Dice Loss and Focal Loss and makes a comparative study between the two. The proposed framework is validated with our in-house dataset consisting of 290 skin biopsy images. Two different experiments are considered. Under the first protocol, only 3-fold cross-validation is done to directly compare the current results with the state-of-the-art ones. Next, the performance of the system on a held-out data set is reported. The experimental results show that MICaps improves the state-of-the-art diagnosis performance by 3.27% (maximum) and reduces the number of model parameters by 50%.

Paediatric echocardiography is a standard method for screening congenital heart disease (CHD). The segmentation of paediatric echocardiography is essential for subsequent extraction of clinical parameters and interventional planning. However, it remains a challenging task due to (1) the considerable variation of key anatomic structures, (2) the poor lateral resolution affecting accurate boundary definition, (3) the existence of speckle noise and artefacts in echocardiographic images. In this paper, we propose a novel deep network to address these challenges comprehensively. We first present a dual-path feature extraction module (DP-FEM) to extract rich features via a channel attention mechanism. A high- and low-level feature fusion module (HL-FFM) is devised based on spatial attention, which selectively fuses rich semantic information from high-level features with spatial cues from low-level features. In addition, a hybrid loss is designed to deal with pixel-level misalignment and boundary ambiguities. Based on the segmentation results, we derive key clinical parameters for diagnosis and treatment planning. We extensively evaluate the proposed method on 4,485 two-dimensional (2D) paediatric echocardiograms from 127 echocardiographic videos. The proposed method consistently achieves better segmentation performance than other state-of-the-art methods, whichdemonstratesfeasibility for automatic segmentation and quantitative analysis of paediatric echocardiography. Our code is publicly available at https://github.com/end-of-the-century/Cardiac.

Purpose Information on the brain microstructure can be probed by Diffusion Magnetic Resonance Imaging (dMRI). Neurite Orientation Dispersion and Density Imaging with Diffusivities Assessment (NODDIDA) is one of the simplest microstructural model proposed. However, the estimation of the NODDIDA parameters from clinically plausible dMRI acquisition is ill‐posed, and different parameter sets can describe the same measurements equally well. A few approaches to resolve this problem focused on developing better optimization strategies for this non‐convex optimization. However, this fundamentally does not resolve ill‐posedness. This article introduces a Bayesian estimation framework, which is regularized through knowledge from an extensive dMRI measurement set on a population of healthy adults (henceforth population‐based prior). Methods We reformulate the problem as a Bayesian maximum a posteriori estimation, which includes as a special case previous approach using non‐informative uniform priors. A population‐based prior is estimated from 35 subjects of the MGH Adult Diffusion data (Human Connectome Project), acquired with an extensive acquisition protocol including high b‐values. The accuracy and robustness of different approaches with and without the population‐based prior is tested on subsets of the MGH dataset, and an independent dataset from a clinically comparable scanner, with only clinically plausible dMRI measurements. Results The population‐based prior produced substantially more accurate and robust parameter estimates, compared to the conventional uniform priors, for clinically feasible protocols, without introducing any evident bias. Conclusions The use of the proposed Bayesian population‐based prior can lead to clinically feasible and robust estimation of NODDIDA parameters without changing the acquisition protocol.

In this study, we present a segmentation algorithm based on ray casting and border point detection. The algorithm’s main parameter is the number of emitted rays, which defines the resolution of the object’s boundary. The value of this parameter depends on the shape of the target region. For instance, 8 rays are enough to segment the left ventricle with the average Dice similarity coefficient approximately equal to 85%. Having gathered the data of rays, the training datasets had a relatively high level of class imbalance (up to 90%). To cope with this issue, ensemble-based classifiers used to manage imbalanced datasets such as AdaBoost.M2, RUSBoost, UnderBagging, SMOTEBagging, SMOTEBoost were used for border detection. For estimation of the accuracy and processing time, the proposed algorithm used a cardiac MRI dataset of the University of York and brain tumour dataset of Southern Medical University. The highest Dice similarity coefficients for the heart and brain tumour segmentation, equal to 86.5 ± 6.9% and 89.5 ± 6.7%, respectively, were achieved by the proposed algorithm. The segmentation time of a cardiac frame equals 4.1 ± 2.3 ms and 20.2 ± 23.6 ms for 8 and 64 rays, respectively. Brain tumour segmentation took 5.1 ± 1.1 ms and 16.0 ± 3.0 ms for 8 and 64 rays respectively. By testing the different medical imaging cases, the proposed algorithm is not time-consuming and highly accurate for convex and closed objects. The scalability of the algorithm allows implementing different border detection techniques working in parallel.

Image registration is a fundamental task in multiple medical image analysis applications. With the advent of deep learning, there have been significant advances in algorithmic performance for various computer vision tasks in recent years, including medical image registration. The last couple of years have seen a dramatic increase in the development of deep learning-based medical image registration algorithms. Consequently, a comprehensive review of the current state-of-the-art algorithms in the field is timely, and necessary. This review is aimed at understanding the clinical applications and challenges that drove this innovation, analysing the functionality and limitations of existing approaches, and at providing insights to open challenges and as yet unmet clinical needs that could shape future research directions. To this end, the main contributions of this paper are: (a) discussion of all deep learning-based medical image registration papers published since 2013 with significant methodological and/or functional contributions to the field; (b) analysis of the development and evolution of deep learning-based image registration methods, summarising the current trends and challenges in the domain; and (c) overview of unmet clinical needs and potential directions for future research in deep learning-based medical image registration.

Automatic shape anomaly detection in large-scale imaging data can be useful for screening suboptimal segmentations and pathologies altering the cardiac morphology without intensive manual labour. We propose a deep probabilistic model for local anomaly detection in sequences of heart shapes, modelled as point sets, in a cardiac cycle. A deep recurrent encoder-decoder network captures the spatio-temporal dependencies to predict the next shape in the cycle and thus derive the outlier points that are attributed to excessive deviations from the network prediction. A predictive mixture distribution models the inlier and outlier classes via Gaussian and uniform distributions, respectively. A Gibbs sampling Expectation-Maximisation (EM) algorithm computes soft anomaly scores of the points via the posterior probabilities of each class in the E-step and estimates the parameters of the network and the predictive distribution in the M-step. We demonstrate the versatility of the method using two shape datasets derived from: i) one million biventricular CMR images from 20,000 participants in the UK Biobank (UKB), and ii) routine diagnostic imaging from Multi-Centre, Multi-Vendor, and Multi-Disease Cardiac Image (M&Ms). Experiments show that the detected shape anomalies in the UKB dataset are mostly associated with poor segmentation quality, and the predicted shape sequences show significant improvement over the input sequences. Furthermore, evaluations on U-Net based shapes from the M&Ms dataset reveals that the anomalies are attributable to the underlying pathologies that affect the ventricles. The proposed model can therefore be used as an effective mechanism to sift shape anomalies in large-scale cardiac imaging pipelines for further analysis.

Parkinson's disease (PD) is known as an irreversible neurodegenerative disease that mainly affects the patient's motor system. Early classification and regression of PD are essential to slow down this degenerative process from its onset. In this article, a novel adaptive unsupervised feature selection approach is proposed by exploiting manifold learning from longitudinal multimodal data. Classification and clinical score prediction are performed jointly to facilitate early PD diagnosis. Specifically, the proposed approach performs united embedding and sparse regression, which can determine the similarity matrices and discriminative features adaptively. Meanwhile, we constrain the similarity matrix among subjects and exploit the l2,p norm to conduct sparse adaptive control for obtaining the intrinsic information of the multimodal data structure. An effective iterative optimization algorithm is proposed to solve this problem. We perform abundant experiments on the Parkinson's Progression Markers Initiative (PPMI) data set to verify the validity of the proposed approach. The results show that our approach boosts the performance on the classification and clinical score regression of longitudinal data and surpasses the state-of-the-art approaches.

Deep segmentation models that generalize to images with unknown appearance are important for real-world medical image analysis. Retraining models leads to high latency and complex pipelines, which are impractical in clinical settings. The situation becomes more severe for ultrasound image analysis because of their large appearance shifts. In this paper, we propose a novel method for robust segmentation under unknown appearance shifts. Our contribution is three-fold. First, we advance a one-stage plug-and-play solution by embedding hierarchical style transfer units into a segmentation architecture. Our solution can remove appearance shifts and perform segmentation simultaneously. Second, we adopt Dynamic Instance Normalization to conduct precise and dynamic style transfer in a learnable manner, rather than previously fixed style normalization. Third, our solution is fast and lightweight for routine clinical adoption. Given 400*400 image input, our solution only needs an additional 0.2ms and 1.92M FLOPs to handle appearance shifts compared to the baseline pipeline. Extensive experiments are conducted on a large dataset from three vendors demonstrate our proposed method enhances the robustness of deep segmentation models., Comment: Accepted by IEEE ISBI 2021

In large studies involving multi protocol Magnetic Resonance Imaging (MRI), it can occur to miss one or more sub-modalities for a given patient owing to poor quality (e.g. imaging artifacts), failed acquisitions, or hallway interrupted imaging examinations. In some cases, certain protocols are unavailable due to limited scan time or to retrospectively harmonise the imaging protocols of two independent studies. Missing image modalities pose a challenge to segmentation frameworks as complementary information contributed by the missing scans is then lost. In this paper, we propose a novel model, Multi-modal Gaussian Process Prior Variational Autoencoder (MGP-VAE), to impute one or more missing sub-modalities for a patient scan. MGP-VAE can leverage the Gaussian Process (GP) prior on the Variational Autoencoder (VAE) to utilize the subjects/patients and sub-modalities correlations. Instead of designing one network for each possible subset of present sub-modalities or using frameworks to mix feature maps, missing data can be generated from a single model based on all the available samples. We show the applicability of MGP-VAE on brain tumor segmentation where either, two, or three of four sub-modalities may be missing. Our experiments against competitive segmentation baselines with missing sub-modality on BraTS’19 dataset indicate the effectiveness of the MGP-VAE model for segmentation tasks.

Nodule segmentation from breast ultrasound images is challenging yet essential for the diagnosis. Weakly-supervised segmentation (WSS) can help reduce time-consuming and cumbersome manual annotation. Unlike existing weakly-supervised approaches, in this study, we propose a novel and general WSS framework called Flip Learning, which only needs the box annotation. Specifically, the target in the label box will be erased gradually to flip the classification tag, and the erased region will be considered as the segmentation result finally. Our contribution is three-fold. First, our proposed approach erases superpixel level using a Multi-agent Reinforcement Learning framework to exploit the prior boundary knowledge and accelerate the learning process. Second, we design two rewards: classification score and intensity distribution reward, to avoid under- and over-segmentation, respectively. Third, we adopt a coarse-to-fine learning strategy to reduce the residual errors and improve the segmentation performance. Extensively validated on a large dataset, our proposed approach achieves competitive performance and shows great potential to narrow the gap between fully-supervised and weakly-supervised learning.

Prospective studies with linked image and genetic data, such as the UK Biobank (UKB), provide an unprecedented opportunity to extract knowledge on the genetic basis of image-derived phenotypes. However, the extent of phenotypes tested within so-called genome-wide association studies (GWAS) is usually limited to handcrafted features, where the main limitation to proceed otherwise is the high dimensionality of both the imaging and genetic data. Here, we propose an approach where the phenotyping is performed in an unsupervised manner, via autoencoders that operate on image-derived 3D meshes. Therefore, the latent variables produced by the encoder condense the information related to the geometry of the biologic structure of interest. The network’s training proceeds in two steps: the first is genotype-agnostic and the second enforces an association with a set of genetic markers selected via GWAS on the intermediate latent representation. This genotype-dependent optimisation procedure allows the refinement of the phenotypes produced by the autoencoder to better understand the effect of the genetic markers encountered. We tested and validated our proposed method on left-ventricular meshes derived from cardiovascular magnetic resonance images from the UKB, leading to the discovery of novel genetic associations that, to the best of our knowledge, had not been yet reported in the literature on cardiac phenotypes.

3D ultrasound (US) is widely used for its rich diagnostic information. However, it is criticized for its limited field of view. 3D freehand US reconstruction is promising in addressing the problem by providing broad range and freeform scan. The existing deep learning based methods only focus on the basic cases of skill sequences, and the model relies on the training data heavily. The sequences in real clinical practice are a mix of diverse skills and have complex scanning paths. Besides, deep models should adapt themselves to the testing cases with prior knowledge for better robustness, rather than only fit to the training cases. In this paper, we propose a novel approach to sensorless freehand 3D US reconstruction considering the complex skill sequences. Our contribution is three-fold. First, we advance a novel online learning framework by designing a differentiable reconstruction algorithm. It realizes an end-to-end optimization from section sequences to the reconstructed volume. Second, a self-supervised learning method is developed to explore the context information that reconstructed by the testing data itself, promoting the perception of the model. Third, inspired by the effectiveness of shape prior, we also introduce adversarial training to strengthen the learning of anatomical shape prior in the reconstructed volume. By mining the context and structural cues of the testing data, our online learning methods can drive the model to handle complex skill sequences. Experimental results on developmental dysplasia of the hip US and fetal US datasets show that, our proposed method can outperform the start-of-the-art methods regarding the shift errors and path similarities.

Standard plane recognition plays an important role in prenatal ultrasound (US) screening. Automatically recognizing the standard plane along with the corresponding anatomical structures in US image can not only facilitate US image interpretation but also improve diagnostic efficiency. In this study, we build a novel multi-label learning (MLL) scheme to identify multiple standard planes and corresponding anatomical structures of fetus simultaneously. Our contribution is three-fold. First, we represent the class correlation by word embeddings to capture the fine-grained semantic and latent statistical concurrency. Second, we equip the MLL with a graph convolutional network to explore the inner and outer relationship among categories. Third, we propose a novel cluster relabel-based contrastive learning algorithm to encourage the divergence among ambiguous classes. Extensive validation was performed on our large in-house dataset. Our approach reports the highest accuracy as 90.25\(\%\) for standard planes labeling, 85.59\(\%\) for planes and structures labeling and mAP as 94.63\(\%\). The proposed MLL scheme provides a novel perspective for standard plane recognition and can be easily extended to other medical image classification tasks.

Image registration aims to establish spatial correspondence across pairs, or groups of images, and is a cornerstone of medical image computing and computer-assisted-interventions. Currently, most deep learning-based registration methods assume that the desired deformation fields are globally smooth and continuous, which is not always valid for real-world scenarios, especially in medical image registration (e.g. cardiac imaging and abdominal imaging). Such a global constraint can lead to artefacts and increased errors at discontinuous tissue interfaces. To tackle this issue, we propose a weakly-supervised Deep Discontinuity-preserving Image Registration network (DDIR), to obtain better registration performance and realistic deformation fields. We demonstrate that our method achieves significant improvements in registration accuracy and predicts more realistic deformations, in registration experiments on cardiac magnetic resonance (MR) images from UK Biobank Imaging Study (UKBB), than state-of-the-art approaches.

The performance of deep segmentation models often degrades due to distribution shifts in image intensities between the training and test data sets. This is particularly pronounced in multi-centre studies involving data acquired using multi-vendor scanners, with variations in acquisition protocols. It is challenging to address this degradation because the shift is often not known a priori and hence difficult to model. We propose a novel framework to ensure robust segmentation in the presence of such distribution shifts. Our contribution is three-fold. First, inspired by the spirit of curriculum learning, we design a novel style curriculum to train the segmentation models using an easy-to-hard mode. A style transfer model with style fusion is employed to generate the curriculum samples. Gradually focusing on complex and adversarial style samples can significantly boost the robustness of the models. Second, instead of subjectively defining the curriculum complexity, we adopt an automated gradient manipulation method to control the hard and adversarial sample generation process. Third, we propose the Local Gradient Sign strategy to aggregate the gradient locally and stabilise training during gradient manipulation. The proposed framework can generalise to unknown distribution without using any target data. Extensive experiments on the public M&Ms Challenge dataset demonstrate that our proposed framework can generalise deep models well to unknown distributions and achieve significant improvements in segmentation accuracy.

Automated detection of curvilinear structures, e.g., blood vessels or nerve fibres, from medical and biomedical images is a crucial early step in automatic image interpretation associated to the management of many diseases. Precise measurement of the morphological changes of these curvilinear organ structures informs clinicians for understanding the mechanism, diagnosis, and treatment of e.g. cardiovascular, kidney, eye, lung, and neurological conditions. In this work, we propose a generic and unified convolution neural network for the segmentation of curvilinear structures and illustrate in several 2D/3D medical imaging modalities. We introduce a new curvilinear structure segmentation network (CS2-Net), which includes a self-attention mechanism in the encoder and decoder to learn rich hierarchical representations of curvilinear structures. Two types of attention modules - spatial attention and channel attention - are utilized to enhance the inter-class discrimination and intra-class responsiveness, to further integrate local features with their global dependencies and normalization, adaptively. Furthermore, to facilitate the segmentation of curvilinear structures in medical images, we employ a 1×3 and a 3×1 convolutional kernel to capture boundary features. Besides, we extend the 2D attention mechanism to 3D to enhance the network's ability to aggregate depth information across different layers/slices. The proposed curvilinear structure segmentation network is thoroughly validated using both 2D and 3D images across six different imaging modalities. Experimental results across nine datasets show the proposed method generally outperforms other state-of-the-art algorithms in various metrics.

Graph-based groupwise registration methods are widely used in atlas construction. Given a group of images, a graph is built whose nodes represent the images, and whose edges represent a geodesic path between two nodes. The distribution of images on an image manifold is explored through edge traversal in a graph. The final atlas is a mean image at the population center of the distribution on the manifold. The procedure of warping all images to the mean image turns to dynamic graph shrinkage in which nodes become closer to each other. Most conventional groupwise registration frameworks construct and shrink a graph without considering the local distribution of images on the dataset manifold and the local structure variations between image pairs. Neglecting the local information fundamentally decrease the accuracy and efficiency when population atlases are built for organs with large inter-subject anatomical variabilities. To overcome the problem, this paper proposes a global-local graph shrinkage approach that can generate accurate atlas. A connected graph is constructed automatically based on global similarities across the images to explore the global distribution. A local image distribution obtained by image clustering is used to simplify the edges of the constructed graph. Subsequently, local image similarities refine the deformation estimated through global image similarity for each image warping along the graph edges. Through the image warping, the overall simplified graph shrinks gradually to yield the atlas with respecting both global and local features. The proposed method is evaluated on 61 synthetic and 20 clinical liver datasets, and the results are compared with those of six state-of-the-art groupwise registration methods. The experimental results show that the proposed method outperforms non-global-local method approaches in terms of accuracy.

Graph convolution networks (GCN) have been successfully applied in disease prediction tasks as they capture interactions (i.e., edges and edge weights on the graph) between individual elements. The interactions in existing works are constructed by fusing similarity between imaging information and distance between non-imaging information, whereas disregarding the disease status of those individuals in the training set. Besides, the similarity is being evaluated by computing the correlation distance between feature vectors, which limits prediction performance, especially for predicting significant memory concern (SMC) and mild cognitive impairment (MCI). In this paper, we propose three mechanisms to improve GCN, namely similarity-aware adaptive calibrated GCN (SAC-GCN), for predicting SMC and MCI. First, we design a similarity-aware graph using different receptive fields to consider disease status. The labelled subjects on the graph are only connected with those labelled subjects with the same status. Second, we propose an adaptive mechanism to evaluate similarity. Specifically, we construct initial GCN with evaluating similarity by using traditional correlation distance, then pre-train the initial GCN by using training samples and use it to score all subjects. Then, the difference between these scores replaces correlation distance to update similarity. Last, we devise a calibration mechanism to fuse functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) information into edges. The proposed method is tested on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Experimental results demonstrate that our proposed method is useful to predict disease-induced deterioration and superior to other related algorithms, with a mean classification accuracy of 86.83% in our prediction tasks.

Accurate ventricular volume measurements are the primary indicators of normal/abnor- mal cardiac function and are dependent on the Cardiac Magnetic Resonance (CMR) volumes being complete. However, missing or unusable slices owing to the presence of image artefacts such as respiratory or motion ghosting, aliasing, ringing and signal loss in CMR sequences, significantly hinder accuracy of anatomical and functional cardiac quantification, and recovering from those is insufficiently addressed in population imaging. In this work, we propose a new robust approach, coined Image Imputation Generative Adversarial Network (I2-GAN), to learn key features of cardiac short axis (SAX) slices near missing information, and use them as conditional variables to infer missing slices in the query volumes. In I2-GAN, the slices are first mapped to latent vectors with position features through a regression net. The latent vector corresponding to the desired position is then projected onto the slice manifold, conditioned on intensity features through a generator net. The generator comprises residual blocks with normalisation layers that are modulated with auxiliary slice information, enabling propagation of fine details through the network. In addition, a multi-scale discriminator was implemented, along with a discriminator-based feature matching loss, to further enhance performance and encourage the synthesis of visually realistic slices. Experimental results show that our method achieves significant improvements over the state-of-the-art, in missing slice imputation for CMR, with an average SSIM of 0.872. Linear regression analysis yields good agreement between reference and imputed CMR images for all cardiac measurements, with correlation coefficients of 0.991 for left ventricular volume, 0.977 for left ventricular mass and 0.961 for right ventricular volume.

Salivary gland ultrasonography (SGUS) has shown good potential in the diagnosis of primary Sjögren's syndrome (pSS). However, a series of international studies have reported needs for improvements of the existing pSS scoring procedures in terms of inter/intra observer reliability before being established as standardized diagnostic tools. The present study aims to solve this problem by employing radiomics features and artificial intelligence (AI) algorithms to make the pSS scoring more objective and faster compared to human expert scoring. The assessment of AI algorithms was performed on a two-centric cohort, which included 600 SGUS images (150 patients) annotated using the original SGUS scoring system proposed in 1992 for pSS. For each image, we extracted 907 histogram-based and descriptive statistics features from segmented salivary glands. Optimal feature subsets were found using the genetic algorithm based wrapper approach. Among the considered algorithms (seven classifiers and five regressors), the best preforming was the multilayer perceptron (MLP) classifier (κ = 0.7). The MLP over-performed average score achieved by the clinicians (κ = 0.67) by the considerable margin, whereas its reliability was on the level of human intra-observer variability (κ = 0.71). The presented findings indicate that the continuously increasing HarmonicSS cohort will enable further advancements in AI-based pSS scoring methods by SGUS. In turn, this may establish SGUS as an effective noninvasive pSS diagnostic tool, with the final goal to supplement current diagnostic tests. ispartof: IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS vol:24 issue:3 pages:835-843 ispartof: location:United States status: published

Significant memory concern (SMC) is the earlier stage of mild cognitive impairment (MCI), and its early treatment is quite vital to delay further disease-induced deterioration. To predict the deterioration, graph convolution network (GCN) with current adjacency matrix still suffers from limited prediction performance due to their subtle difference and obscure features. For this reason, we propose a similarity-aware adaptive calibrated GCN (SAC-GCN), which can combine functional and structural information to predict SMC and MCI. We utilize an adaptive calibration mechanism to construct a data-driven adjacency matrix. Specifically, we first design a similarity-aware graph using different receptive fields to consider the disease statuses. Namely, the labeled subjects are only connected with those subjects who have the same status in the convolution operation. Then we compute more accurate weights in graph edges from functional and structural scores. Current edge weights are used to construct an initial graph and pre-train the GCN. Based on the pre-trained GCN, the differences between scores replace the traditional correlation distances to evaluate edge weights. Lastly, we devise a calibration technique to fuse functional and structural information for edge weighting. The proposed method is tested on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset. The experimental results demonstrate that our proposed method is effective to predict disease-induced deterioration and superior over other related algorithm.

Labelling data is expensive and time consuming especially for domains such as medical imaging that contain volumetric imaging data and require expert knowledge. Exploiting a larger pool of labeled data available across multiple centers, such as in federated learning, has also seen limited success since current deep learning approaches do not generalize well to images acquired with scanners from different manufacturers. We aim to address these problems in a common, learning-based image simulation framework which we refer to as Federated Simulation. We introduce a physics-driven generative approach that consists of two learnable neural modules: 1) a module that synthesizes 3D cardiac shapes along with their materials, and 2) a CT simulator that renders these into realistic 3D CT Volumes, with annotations. Since the model of geometry and material is disentangled from the imaging sensor, it can effectively be trained across multiple medical centers. We show that our data synthesis framework improves the downstream segmentation performance on several datasets.

Ultrasound (US) image segmentation embraced its significant improvement in deep learning era. However, the lack of sharp boundaries in US images still remains an inherent challenge for segmentation. Previous methods often resort to global context, multi-scale cues or auxiliary guidance to estimate the boundaries. It is hard for these methods to approach pixel-level learning for fine-grained boundary generating. In this paper, we propose a novel and effective framework to improve boundary estimation in US images. Our work has three highlights. First, we propose to formulate the boundary estimation as a rendering task, which can recognize ambiguous points (pixels/voxels) and calibrate the boundary prediction via enriched feature representation learning. Second, we introduce point-wise contrastive learning to enhance the similarity of points from the same class and contrastively decrease the similarity of points from different classes. Boundary ambiguities are therefore further addressed. Third, both rendering and contrastive learning tasks contribute to consistent improvement while reducing network parameters. As a proof-of-concept, we performed validation experiments on a challenging dataset of 86 ovarian US volumes. Results show that our proposed method outperforms state-of-the-art methods and has the potential to be used in clinical practice.

Subjective cognitive decline (SCD) is an early stage of mild cognitive impairment (MCI) and may represent the first symptom manifestation of Alzheimer’s disease (AD). Early diagnosis of MCI is important because early identification and intervention can delay or even reverse the progression of this disease. This paper proposes an automatic diagnostic framework for SCD and MCI. Specifically, we design a new multi-task learning model to integrate neuroimaging functional and structural connectivity in a predictive framework. We construct a functional brain network by sparse low-rank brain network estimation methods, and a structural brain network is constructed using fiber bundle tracking. Subsequently, we use multi-task learning methods to select features for integrated functional and structural connections, the importance of each task and the balance between both modalities are automatically learned. By integrating both functional and structural information, the most discriminative features of the disease are obtained for diagnosis. The experiments on the dataset show that our proposed method achieves good performance and is superior to the traditional algorithms. In addition, the proposed method can identify the most discriminative brain regions and connections. These results follow current clinical findings and add new findings for disease detection and future medical analysis.

Stroke is the third most common cause of death and a major contributor to long-term disability worldwide. Severe stroke is most often caused by the rupture of a cerebral aneurysm, a weakened area in a blood vessel. The detection and quantification of cerebral aneurysms are essential for the prevention and treatment of aneurysmal rupture and cerebral infarction. Here, we propose a novel aneurysm detection method in a three-dimensional (3D) cerebrovascular model based on convolutional neural networks (CNNs). The multiview method is used to obtain a sequence of 2D images on the cerebral vessel branch model. The pretrained CNN is used with transfer learning to overcome the small training sample problem. The data augmentation strategy with rotation, mirroring and flipping helps improve the performance dramatically, particularly on our small datasets. The hyperparameter of the view number is determined in the task. We have applied the labeling task on 56 3D mesh models with aneurysms (positive) and 65 models without aneurysms (negative). The average accuracy of individual projected images is 87.86%, while that of the model is 93.4% with the best view number. The framework is highly effective with quick training efficiency that can be widely extended to detect other organ anomalies.

3D ultrasound (US) is widely used due to its rich diagnostic information, portability and low cost. Automated standard plane (SP) localization in US volume not only improves efficiency and reduces user-dependence, but also boosts 3D US interpretation. In this study, we propose a novel Multi-Agent Reinforcement Learning (MARL) framework to localize multiple uterine SPs in 3D US simultaneously. Our contribution is two-fold. First, we equip the MARL with a one-shot neural architecture search (NAS) module to obtain the optimal agent for each plane. Specifically, Gradient-based search using Differentiable Architecture Sampler (GDAS) is employed to accelerate and stabilize the training process. Second, we propose a novel collaborative strategy to strengthen agents’ communication. Our strategy uses recurrent neural network (RNN) to learn the spatial relationship among SPs effectively. Extensively validated on a large dataset, our approach achieves the accuracy of 7.05\(^{\circ }\)/2.21 mm, 8.62\(^{\circ }\)/2.36 mm and 5.93\(^{\circ }\)/0.89 mm for the mid-sagittal, transverse and coronal plane localization, respectively. The proposed MARL framework can significantly increase the plane localization accuracy and reduce the computational cost and model size.

In this letter, we describe a novel framework for planning and executing semi-autonomous tissue retraction in minimally invasive robotic surgery. The approach is aimed at removing tissue flaps or connective tissue from the surgical area autonomously, thus exposing the underlying anatomical structures. First, a deep neural network is used to analyse the endoscopic image and detect candidate tissue flaps obstructing the surgical field. A procedural algorithm for planning and executing the retraction gesture is then developed from extended discussions with clinicians. Experimental validation, carried out on a DaVinci Research Kit, shows an average 25% increase of the visible background after retraction. Another significant contribution of this letter is a dataset containing 1,080 labelled surgical stereo images and the associated depth maps, representing tissue flaps in different scenarios. The work described in this letter is a fundamental step towards the autonomous execution of tissue retraction, and the first example of simultaneous use of deep learning and procedural algorithms. The same framework could be applied to a wide range of autonomous tasks, such as debridement and placement of laparoscopic clips.

Multi-modality medical imaging is increasingly used for comprehensive assessment of complex diseases in either diagnostic examinations or as part of medical research trials. Different imaging modalities provide complementary information about living tissues. However, multi-modal examinations are not always possible due to adversary factors, such as patient discomfort, increased cost, prolonged scanning time, and scanner unavailability. In additionally, in large imaging studies, incomplete records are not uncommon owing to image artifacts, data corruption or data loss, which compromise the potential of multi-modal acquisitions. In this paper, we propose a weakly coupled and geometry co-regularized joint dictionary learning method to address the problem of cross-modality synthesis while considering the fact that collecting the large amounts of training data is often impractical. Our learning stage requires only a few registered multi-modality image pairs as training data. To employ both paired images and a large set of unpaired data, a cross-modality image matching criterion is proposed. Then, we propose a unified model by integrating such a criterion into the joint dictionary learning and the observed common feature space for associating cross-modality data for the purpose of synthesis. Furthermore, two regularization terms are added to construct robust sparse representations. Our experimental results demonstrate superior performance of the proposed model over state-of-the-art methods.

Accurately determining the spatial relationship between the pelvis and acetabulum is challenging due to their inherently complex three-dimensional (3D) anatomy. A standardized 3D pelvic coordinate system (PCS) and the precise assessment of acetabular orientation would enable the relationship to be determined. We present a surface-based method to establish a reliable PCS and develop software for semi-automatic measurement of acetabular spatial parameters. Vertices on the acetabular rim were manually extracted as an eigenpoint set after 3D models were imported into the software. A reliable PCS consisting of the anterior pelvic plane, midsagittal pelvic plane, and transverse pelvic plane was then computed by iteration on mesh data. A spatial circle was fitted as a succinct description of the acetabular rim. Finally, a series of mutual spatial parameters between the pelvis and acetabulum were determined semi-automatically, including the center of rotation, radius, and acetabular orientation. Pelvic models were reconstructed based on high-resolution computed tomography images. Inter- and intra-rater correlations for measurements of mutual spatial parameters were almost perfect, showing our method affords very reproducible measurements. The approach will thus be useful for analyzing anatomic data and has potential applications for preoperative planning in individuals receiving total hip arthroplasty.

Early diagnosis and intervention of mild cognitive impairment (MCI) and its early stage (i.e., subjective cognitive decline (SCD)) is able to delay or reverse the disease progression. However, discrimination between SCD, MCI and healthy subjects accurately remains challenging. This paper proposes an auto-weighted centralised multi-task (AWCMT) learning framework for differential diagnosis of SCD and MCI. AWCMT is based on structural and functional connectivity information inferred from magnetic resonance imaging (MRI). To be specific, we devise a novel multi-task learning algorithm to combine neuroimaging functional and structural connective information. We construct a functional brain network through a sparse and low-rank machine learning method, and also a structural brain network via fibre bundle tracking. Those two networks are constructed separately and independently. Multi-task learning is then used to identify features integration of functional and structural connectivity. Hence, we can learn each task's significance automatically in a balanced way. By combining the functional and structural information, the most informative features of SCD and MCI are obtained for diagnosis. The extensive experiments on the public and self-collected datasets demonstrate that the proposed algorithm obtains better performance in classifying SCD, MCI and healthy people than traditional algorithms. The newly proposed method has good interpretability as it is able to discover the most disease-related brain regions and their connectivity. The results agree well with current clinical findings and provide new insights into early AD detection based on the multi-modal neuroimaging technique.

Most well-known blind image quality assessment (BIQA) models usually follow a two-stage framework whereby various types of features are first extracted and used as an input to a regressor. The regression algorithm is used to model human perceptual measures based on a training set of distorted images. However, this approach requires an intensive training phase to optimise the regression parameters. In this paper, we overcome this limitation by proposing an alternative BIQA model that predicts image quality using nearest neighbour methods which have virtually zero training cost. The model, termed PATCH based blind Image Quality assessment (PATCH-IQ), has a learning framework that operates at the patch level. This enables PATCH-IQ to provide not only a global image quality estimation but also a local image quality estimation. Based on the assumption that the perceived quality of a distorted image will be best predicted by features drawn from images with the same distortion class, PATCH-IQ also introduces a distortion identification stage in its framework. This enables PATCH-IQ to identify the distortion affecting the image, a property that can be useful for further local processing stages. PATCH-IQ is evaluated on the standard IQA databases, and the provided scores are highly correlated to human perception of image quality. It also delivers competitive prediction accuracy and computational performance in relationship to other state-of-the-art BIQA models.

Segmentation of ovary and follicles from 3D ultrasound (US) is the crucial technique of measurement tools for female infertility diagnosis. Since manual segmentation is time-consuming and operator-dependent, an accurate and fast segmentation method is highly demanded. However, it is challenging for current deep-learning based methods to segment ovary and follicles precisely due to ambiguous boundaries and insufficient annotations. In this paper, we propose a contrastive rendering (C-Rend) framework to segment ovary and follicles with detail-refined boundaries. Furthermore, we incorporate the proposed C-Rend with a semi-supervised learning (SSL) framework, leveraging unlabeled data for better performance. Highlights of this paper include: (1) A rendering task is performed to estimate boundary accurately via enriched feature representation learning. (2) Point-wise contrastive learning is proposed to enhance the similarity of intra-class points and contrastively decrease the similarity of inter-class points. (3) The C-Rend plays a complementary role for the SSL framework in uncertainty-aware learning, which could provide reliable supervision information and achieve superior segmentation performance. Through extensive validation on large in-house datasets with partial annotations, our method outperforms state-of-the-art methods in various evaluation metrics for both the ovary and follicles.

Segmentation of the left and right ventricles in cardiac MRI (Magnetic Resonance Imaging) is a prerequisite step for evaluating global and regional cardiac function. This work presents a novel and robust schema for MRI segmentation by combining the advantages of deep learning localization and 3D-ASM (3D Active Shape Model) restriction without any user interaction. Three fundamental techniques are exploited: (1) manual 2D contours are used to build distance maps to get 3D ground truth shape, (2) derived right ventricle points are employed to rotate the coarse initial shape for a refined bi-ventricle initial estimation, (3) segmentation results from deep learning are utilised to build distance maps for the 3D-ASM matching process to help image intensity modelling. The datasets used for experimenting the cine MRI data are 1000 cases from UK Biobank, 500 subjects are selected to train CNN (Convolution Neural Network) parameters, and the remaining 500 cases are adopted for validation. Specifically, cases are used to rebuild point distribution and image intensity models, and also utilized to train CNN. In addition, the left 500 cases are used to perform the validation experiments. For the segmentation of the RV (Right Ventricle) endocardial contour, LV (Left Ventricle) endo- and epicardial contours, overlap, Jaccard similarity index, Point-to-surface errors and cardiac functional parameters are calculated. Experimental results show that the proposed method has advantages over the previous approaches.

High-resolution (HR), isotropic cardiac Magnetic Resonance (MR) cine imaging is challenging since it requires long acquisition and patient breath-hold times. Instead, 2D balanced steady-state free precession (SSFP) sequence is widely used in clinical routine. However, it produces highly-anisotropic image stacks, with large through-plane spacing that can hinder subsequent image analysis. To resolve this, we propose a novel, robust adversarial learning super-resolution (SR) algorithm based on conditional generative adversarial nets (GANs), that incorporates a state-of-the-art optical flow component to generate an auxiliary image to guide image synthesis. The approach is designed for real-world clinical scenarios and requires neither multiple low-resolution (LR) scans with multiple views, nor the corresponding HR scans, and is trained in an end-to-end unsupervised transfer learning fashion. The designed framework effectively incorporates visual properties and relevant structures of input images and can synthesise 3D isotropic, anatomically plausible cardiac MR images, consistent with the acquired slices. Experimental results show that the proposed SR method outperforms several state-of-the-art methods both qualitatively and quantitatively. We show that subsequent image analyses including ventricle segmentation, cardiac quantification, and non-rigid registration can benefit from the super-resolved, isotropic cardiac MR images, to produce more accurate quantitative results, without increasing the acquisition time. The average Dice similarity coefficient (DSC) for the left ventricular (LV) cavity and myocardium are 0.95 and 0.81, respectively, between real and synthesised slice segmentation. For non-rigid registration and motion tracking through the cardiac cycle, the proposed method improves the average DSC from 0.75 to 0.86, compared to the original resolution images. ispartof: MEDICAL IMAGE ANALYSIS vol:71 ispartof: location:Netherlands status: published

Background and objective: To address the increasing need for collaborative endeavours within the Virtual Physiological Human (VPH) community, the VPH-Share collaborative cloud platform allows researchers to expose and share sequences of complex biomedical processing tasks in the form of computational workflows. The Taverna Workflow System is a very popular tool for orchestrating complex biomedical & bioinformatics processing tasks in the VPH community. This paper describes the VPH-Share components that support the building and execution of Taverna workflows, and explains how they interact with other VPH-Share components to improve the capabilities of the VPH-Share platform.\ud \ud \ud \ud Methods: Taverna workflow support is delivered by the Atmosphere cloud management platform and the VPH-Share Taverna plugin. These components are explained in detail, along with the two main procedures that were developed to enable this seamless integration: workflow composition and execution.\ud \ud \ud \ud Results: 1) Seamless integration of VPH-Share with other components and systems. 2) Extended range of different tools for workflows. 3) Successful integration of scientific workflows from other VPH projects. 4) Execution speed improvement for medical applications.\ud \ud \ud \ud Conclusion: The presented workflow integration provides VPH-Share users with a wide range of different possibilities to compose and execute workflows, such as desktop or online composition, online batch execution, multithreading, remote execution, etc. The specific advantages of each supported tool are presented, as are the roles of Atmosphere and the VPH-Share plugin within the VPH-Share project. The combination of the VPH-Share plugin and Atmosphere engenders the VPH-Share infrastructure with far more flexible, powerful and usable capabilities for the VPH-Share community. As both components can continue to evolve and improve independently, we acknowledge that further improvements are still to be developed and will be described.

Numerous problems of relevance in physiology and biomechanics, have at their core, the presence of a deformable solid matrix which experiences flow-induced strain. Often, this fluid–structure interaction (FSI) is directed the opposite way, i.e. it is solid deformation that creates flow, with the heart being the most prominent example. In many cases, this interaction of fluid and solid is genuinely bidirectional and strongly coupled, with solid deformation inducing flow and fluid pressure deforming the solid. Although an FSI problem, numerous cases in biomechanics are not tractable via the traditional FSI methodologies: in the internal flows that are of interest to use, the number and range of fluid passages is so vast that the direct approach of a deterministically defined boundary between fluid and solid is impossible to apply. In these cases, homogenisation and statistical treatment of the material-fluid system is possibly the only way forward. Such homogenisation,quite common to flow-only systems through porous media considerations, is also possible for FSI systems, where the loading is effectively internal to the material. A prominent technique of this type is that of poroelasticity. In this paper, we discuss a class of poroelastic theory techniques that allow for the co-existence of a multitude of – always statistically treated – channels and passages of widely different properties: termed multiple-network poroelasticity (or multicompartmental poroelasticity). This paradigm is particularly suitable for the study of living tissue, that is invariably permeated – perfused – by fluids, often different in nature and across a wide range of scales. Multicompartmental poroelasticity is capable of accounting for bidirectional coupling between the fluids and the solid matrix and allows us to track transport of a multitude of substances together with the deformation of the solid material that this transport gives rise to or is caused by, or both. For the purposes of demonstration, we utilise a complex and physiologically very important system, the human brain (specifically, we target the hippocampus), to exemplify the qualities and efficacy of this methodology during the course of Alzheimer’s Disease. The methodology we present has been implemented through the Finite Element Method, in a general manner, allowing for the co-existence of an arbitrary number of compartments. For the applications used in this paper to exemplify the method, a four-compartment implementation is used. A unified pipeline is used on a cohort of 35 subjects to provide statistically meaningful insight into the underlying mechanisms of the neurovascular unit (NVU) in the hippocampus, and to ascertain whether physical activity would have an influence in both swelling and drainage by taking into account both the scaled strain field and the proportion of perfused blood injected into the brain tissue. A key result garnered from his study is the statistically significant differences in right hemisphere hippocampal NVU swelling between males in the control group and females with mild cognitive impairment during high and low activity states.

Detection of early stages of Alzheimer's disease (AD) (i.e., mild cognitive impairment (MCI)) is important to maximize the chances to delay or prevent progression to AD. Brain connectivity networks inferred from medical imaging data have been commonly used to distinguish MCI patients from normal controls (NC). However, existing methods still suffer from limited performance, and classification remains mainly based on single modality data. This paper proposes a new model to automatically diagnosing MCI (early MCI (EMCI) and late MCI (LMCI)) and its earlier stages (i.e., significant memory concern (SMC)) by combining low-rank self-calibrated functional brain networks and structural brain networks for joint multi-task learning. Specifically, we first develop a new functional brain network estimation method. We introduce data quality indicators for self-calibration, which can improve data quality while completing brain network estimation, and perform correlation analysis combined with low-rank structure. Second, functional and structural connected neuroimaging patterns are integrated into our multi-task learning model to select discriminative and informative features for fine MCI analysis. Different modalities are best suited to undertake distinct classification tasks, and similarities and differences among multiple tasks are best determined through joint learning to determine most discriminative features. The learning process is completed by non-convex regularizer, which effectively reduces the penalty bias of trace norm and approximates the original rank minimization problem. Finally, the most relevant disease features classified using a support vector machine (SVM) for MCI identification. Experimental results show that our method achieves promising performance with high classification accuracy and can effectively discriminate between different sub-stages of MCI.

Background and objective:The detection of abnormalities such as lesions or leakage from retinal images is an important health informatics task for automated early diagnosis of diabetic and malarial retinopathy or other eye diseases, in order to prevent blindness and common systematic conditions. In this work, we propose a novel retinal lesion detection method by adapting the concepts of saliency. Methods :Retinal images are firstly segmented as superpixels, two new saliency feature representations: uniqueness and compactness, are then derived to represent the superpixels. The pixel level saliency is then estimated from these superpixel saliency values via a bilateral filter. These extracted saliency features form a matrix for low-rank analysis to achieve saliency detection. The precise contour of a lesion is finally extracted from the generated saliency map after removing confounding structures such as blood vessels, the optic disc, and the fovea. The main novelty of this method is that it is an effective tool for detecting different abnormalities at pixel-level from different modalities of retinal images, without the need to tune parameters. Results:To evaluate its effectiveness, we have applied our method to seven public datasets of diabetic and malarial retinopathy with four different types of lesions: exudate, hemorrhage, microaneurysms, and leakage. The evaluation was undertaken at pixel-level, lesion-level, or image-level according to ground truth availability in these datasets. Conclusions:The experimental results show that the proposed method outperforms existing state-of-the-art ones in applicability, effectiveness, and accuracy.

For precision medicine to be implemented through the lens of in silico technology, it is imperative that biophysical research workflows offer insight into treatments that are specific to a particular illness and to a particular subject. The boundaries of precision medicine can be extended using multiscale, biophysics-centred workflows that consider the fundamental underpinnings of the constituents of cells and tissues and their dynamic environments. Utilising numerical techniques that can capture the broad spectrum of biological flows within complex, deformable and permeable organs and tissues is of paramount importance when considering the core prerequisites of any state-of-the-art precision medicine pipeline. In this work, a succinct breakdown of two precision medicine pipelines developed within two Virtual Physiological Human (VPH) projects are given. The first workflow is targeted on the trajectory of Alzheimer's Disease, and caters for novel hypothesis testing through a multicompartmental poroelastic model which is integrated with a high throughput imaging workflow and subject-specific blood flow variability model. The second workflow gives rise to the patient specific exploration of Aortic Dissections via a multi-scale and compliant model, harnessing imaging, computational fluid-dynamics (CFD) and dynamic boundary conditions. Results relating to the first workflow include some core outputs of the multiporoelastic modelling framework, and the representation of peri-arterial swelling and peri-venous drainage solution fields. The latter solution fields were statistically analysed for a cohort of thirty-five subjects (stratified with respect to disease status, gender and activity level). The second workflow allowed for a better understanding of complex aortic dissection cases utilising both a rigid-wall model informed by minimal and clinically common datasets as well as a moving-wall model informed by rich datasets. ispartof: Morphologie vol:103 issue:343 pages:148-160 ispartof: location:France status: published

In the diagnosis and interventional treatment of coronary artery disease, the 3D[Formula: see text]time reconstruction of the coronary artery on the basis of x-ray angiographic image sequences can provide dynamic structural information. The synchronization of cardiac phases in the sequences is essential for minimizing the influence of cardiorespiratory motion and realizing precise 3D[Formula: see text]time reconstruction. Key points are initially extracted from the first image of a sequence. Matching grid points between consecutive images in the sequence are extracted by a multi-layer matching strategy. Then deep motion tracking (DMT) of key points is achieved by local deformation based on the neighboring grid points of key points. The local deformation is optimized by the Random sample consensus (RANSAC) algorithm. Then, a simple harmonic motion (SHM) model is utilized to distinguish cardiac motion from other motion sources (e.g. respiratory, patient movement, etc). Next, the signal which is composed of cardiac motions is filtered by a band-pass filter to reconstruct the cardiac phases. Finally, the synchronization of cardiac phases from different imaging angles is realized by a piece-wise linear transformation. The proposed method was evaluated using clinical x-ray angiographic image sequences from 13 patients. [Formula: see text] matching points can be accurately computed by the DMT method. The mean peak temporal distance (MPTD) between the reconstructed cardiac phases and the electrocardiograph signal is [Formula: see text] s. The correlation between the cardiac phases of the same patient is over [Formula: see text]. Compared with three other state-of-the-art methods, the proposed method accurately reconstructs and synchronizes the cardiac phases from different sequences of the same patient. The proposed DMT method is robust and highly effective in synchronizing cardiac phases of angiographic image sequences captured from different imaging angles.

Objective: Automatic vascular enhancement in X-ray cineangiography is of crucial interest, for instance, for better visualizing and quantifying coronary arteries in diagnostic and interventional procedures. Methods: A novel patch-based adaptive background subtraction method (PABSM) is proposed automatically enhancing vessels in coronary X-ray cineangiography. First, pixels in the cineangiogram are described by the vesselness and Gabor features. Second, a classifier is utilized to separate the cineangiogram into the rough vascular and non-vascular region. Dilation is applied to the classified binary image to include more vascular region. Third, a patch-based background synthesis is utilized to fill the removed vascular region. Results: A database containing 320 cineangiograms of 175 patients was collected, and then an interventional cardiologist annotated all vascular structures. The performance of PABSM is compared with six state-of-the-art vascular enhancement methods regarding the precision–recall curve and C-value. The area under the precision–recall curve is $0.7133$ , and the C-value is $0.9659$ . Conclusion: PABSM can automatically enhance the coronary artery in the cineangiograms. It preserves the integrity of vascular topological structures, particularly in complex vascular regions, and removes noise caused by the non-uniform gray-level distribution in the cineangiogram. Significance: PABSM can avoid the motion artifacts and it eases the subsequent vascular segmentation, which is crucial for the diagnosis and interventional procedures of coronary artery diseases.