Search

Filter bank multicarrier with offset quadrature amplitude modulation (FBMC/OQAM) is considered as a powerful supplementary waveform for future wireless communications. However, FBMC systems have the same high peak-to-average power ratio (PAPR) problem as other multi-carrier systems, and the PAPR reduction methods designed for orthogonal frequency division multiplexing (OFDM) systems cannot be directly applied to FBMC systems due to the unique overlapping structure of FBMC signals. Therefore, some PAPR suppression schemes tailored to FBMC systems have been proposed, but their computational complexity is prohibitively high for some practical applications. In this paper, we propose a conversion vector-based low-complexity dispersive selection mapping (DSLM) scheme, called the C-DSLM scheme, to reduce the PAPR of FBMC signals. In the C-DSLM scheme, a series of conversion vectors is first carefully designed for FBMC signals with overlapping structures, and then candidate signals are generated by multiplying the original signal by the cyclic shift of the conversion vectors with only a few nonzero elements. The complexity evaluation and simulation results show that the C-DSLM scheme’s PAPR suppression performance is similar to that of the DSLM scheme with computational complexity is only approximately 10% of that of the DSLM scheme. Our proposed C-DSLM scheme is highly efficient and strongly applicable.

Modulation recognition is a major task in many wireless communication systems including cognitive radio and signal reconnaissance. The diversification of modulation schemes and the increased complexity of the channel environment put higher requirements on the correct identification of modulated signals. Deep learning (DL) is considered as a potential solution to solve these problems due to the superior big data processing and classification capabilities. This paper proposes an efficient digital modulation recognition method based on deep neural network (DNN) model. Furthermore, we present the particle swarm optimization (PSO) algorithm to optimize the number of hidden layer nodes of the DNN so as to solve the problem that the traditional DNN is trapped in local minimum values and the number of hidden layer nodes needs selecting manually. In this paper, we utilize the proposed PSO-DNN method to learn characteristics extracted from the modulated signal added by additive white Gaussian noise (AWGN) and to train the network, which can improve the performance of recognition under the condition of low signal-to-noise ratio (SNR). The experimental results demonstrate that the recognition rate on this algorithm has improved by 9.4% and 8.8% compared with methods that adopt conventional DNN and support vector machine (SVM) when SNR equals 0 and 1 dB, respectively. Besides, another experiment compared with the genetic algorithm (GA) also proves that our proposed algorithm is more effective in optimizing the DNN. The proposed method is easy to be implemented so that it has a broad development prospect in modulation recognition.

Applying machine learning of complex motion phenotypes obtained from cardiac MR images allows more accurate prediction of patient outcomes in pulmonary hypertension., Purpose To determine if patient survival and mechanisms of right ventricular failure in pulmonary hypertension could be predicted by using supervised machine learning of three-dimensional patterns of systolic cardiac motion. Materials and Methods The study was approved by a research ethics committee, and participants gave written informed consent. Two hundred fifty-six patients (143 women; mean age ± standard deviation, 63 years ± 17) with newly diagnosed pulmonary hypertension underwent cardiac magnetic resonance (MR) imaging, right-sided heart catheterization, and 6-minute walk testing with a median follow-up of 4.0 years. Semiautomated segmentation of short-axis cine images was used to create a three-dimensional model of right ventricular motion. Supervised principal components analysis was used to identify patterns of systolic motion that were most strongly predictive of survival. Survival prediction was assessed by using difference in median survival time and area under the curve with time-dependent receiver operating characteristic analysis for 1-year survival. Results At the end of follow-up, 36% of patients (93 of 256) died, and one underwent lung transplantation. Poor outcome was predicted by a loss of effective contraction in the septum and free wall, coupled with reduced basal longitudinal motion. When added to conventional imaging and hemodynamic, functional, and clinical markers, three-dimensional cardiac motion improved survival prediction (area under the receiver operating characteristic curve, 0.73 vs 0.60, respectively; P < .001) and provided greater differentiation according to difference in median survival time between high- and low-risk groups (13.8 vs 10.7 years, respectively; P < .001). Conclusion A machine-learning survival model that uses three-dimensional cardiac motion predicts outcome independent of conventional risk factors in patients with newly diagnosed pulmonary hypertension. Online supplemental material is available for this article.

Due to the lack of large-scale datasets, the prevailing approach in visual sentiment analysis is to leverage models trained for object classification in large datasets like ImageNet. However, objects are sentiment neutral which hinders the expected gain of transfer learning for such tasks. In this work, we propose to overcome this problem by learning a novel sentiment-aligned image embedding that is better suited for subsequent visual sentiment analysis. Our embedding leverages the intricate relation between emojis and images in large-scale and readily available data from social media. Emojis are language-agnostic, consistent, and carry a clear sentiment signal which make them an excellent proxy to learn a sentiment aligned embedding. Hence, we construct a novel dataset of 4 million images collected from Twitter with their associated emojis. We train a deep neural model for image embedding using emoji prediction task as a proxy. Our evaluation demonstrates that the proposed embedding outperforms the popular object-based counterpart consistently across several sentiment analysis benchmarks. Furthermore, without bell and whistles, our compact, effective and simple embedding outperforms the more elaborate and customized state-of-the-art deep models on these public benchmarks. Additionally, we introduce a novel emoji representation based on their visual emotional response which supports a deeper understanding of the emoji modality and their usage on social media., Comment: International Conference on Computer Vision (ICCV 2019) Workshops. Project page and the Visual Smiley Dataset: https://www.cs.utexas.edu/~ziad/emoji_visual_sentiment.html

One of the challenges in display advertising is that the distribution of features and click through rate (CTR) can exhibit large shifts over time due to seasonality, changes to ad campaigns and other factors. The predominant strategy to keep up with these shifts is to train predictive models continuously, on fresh data, in order to prevent them from becoming stale. However, in many ad systems positive labels are only observed after a possibly long and random delay. These delayed labels pose a challenge to data freshness in continuous training: fresh data may not have complete label information at the time they are ingested by the training algorithm. Naive strategies which consider any data point a negative example until a positive label becomes available tend to underestimate CTR, resulting in inferior user experience and suboptimal performance for advertisers. The focus of this paper is to identify the best combination of loss functions and models that enable large-scale learning from a continuous stream of data in the presence of delayed labels. In this work, we compare 5 different loss functions, 3 of them applied to this problem for the first time. We benchmark their performance in offline settings on both public and proprietary datasets in conjunction with shallow and deep model architectures. We also discuss the engineering cost associated with implementing each loss function in a production environment. Finally, we carried out online experiments with the top performing methods, in order to validate their performance in a continuous training scheme. While training on 668 million in-house data points offline, our proposed methods outperform previous state-of-the-art by 3% relative cross entropy (RCE). During online experiments, we observed 55% gain in revenue per thousand requests (RPMq) against naive log loss., Accepted at RecSys '19

Objectives This study used high-resolution 3-dimensional cardiac magnetic resonance to define the anatomical and functional left ventricular (LV) properties associated with increasing systolic blood pressure (SBP) in a drug-naive cohort.

Introduction Hypertrophic cardiomyopathy (HCM) is characterised by great phenotypic diversity and broad spectrum of clinical courses. The genetic, environmental and phenotypic determinants of outcome remain poorly understood. We integrated machine-learning analysis of cardiovascular magnetic resonance (CMR) with computational modelling to define the effects of genetic variation on the heart in both HCM patients and heathy volunteers. Methods Healthy volunteers were recruited at Imperial College London (n=1367) and National Health Centre Singapore (n=754). Patients with HCM were enrolled at the Royal Brompton Hospital (n=622) and National Heart Centre Singapore (n=211). Participants underwent conventional CMR at 1.5 T. Using cardiac atlas and machine learning techniques, CMRs were segmented and co-registered providing statistical models of phenotypic variation. Subjects were sequenced with comprehensive gene panels and using stringent criteria identified as genotype positive (G+), negative (G-) or as carriers of variants of unknown significance (VUS). Results In healthy volunteers, sarcomeric G+variants were associated with increased septal and apical LV wall thickness. In HCM, sarcomeric thin filament G+displayed the mildest global hypertrophy. Sarcomeric thick filament G+variants were associated with asymmetric septal hypertrophy, when compared to G-, VUS and other G+. Conclusion We show that in a healthy population, rare variants in sarcomeric genes are penetrant and associated with increased wall thickness. This has potential clinical implications to the ~0.5% of the population that are carriers. In HCM, distinct patterns of hypertrophy were associated with specific genotypes. We demonstrate that machine-learning analysis of CMRs offers unparalleled insights into the earliest manifestations of cardiomyopathy and mutation-specific pathophysiology.

We propose a new approach for the motion recognition of basketball players based on unit gesture decomposition, which can be wildly used for analysis and statistics of the actions of each player. Meanwhile, we analyze the variation of limb angle and extract the characteristics for immediate actions and continuous actions. In the end, due to the divergence of different limbs, we construct two different classifiers corresponding to the upper and lower limbs respectively and obtain the optimal classifier for motion recognition of basketball players by comparing the output of four most popular machine learning algorithms. Our approach tends to have an adequate performance in practice, for recognizing nine different motions of basketball players with average accuracy over \(98.85\%\).

Convolutional neural networks have enabled accurate image super-resolution in real-time. However, recent attempts to benefit from temporal correlations in video super-resolution have been limited to naive or inefficient architectures. In this paper, we introduce spatio-temporal sub-pixel convolution networks that effectively exploit temporal redundancies and improve reconstruction accuracy while maintaining real-time speed. Specifically, we discuss the use of early fusion, slow fusion and 3D convolutions for the joint processing of multiple consecutive video frames. We also propose a novel joint motion compensation and video super-resolution algorithm that is orders of magnitude more efficient than competing methods, relying on a fast multi-resolution spatial transformer module that is end-to-end trainable. These contributions provide both higher accuracy and temporally more consistent videos, which we confirm qualitatively and quantitatively. Relative to single-frame models, spatio-temporal networks can either reduce the computational cost by 30% whilst maintaining the same quality or provide a 0.2dB gain for a similar computational cost. Results on publicly available datasets demonstrate that the proposed algorithms surpass current state-of-the-art performance in both accuracy and efficiency., Comment: Changes: * Uploaded Vid4 results (footnote 1). * Added references [14, 29] as spatial-transformer prior art. * Fixed typos

In this paper, a compound eye array model which consists of hexagonal dome light receivers arranged on a hemispherical surface for capturing, aiming and tracking laser channel of satellite space communication is proposed and the laser beam aiming algorithm is bought out for angular positioning of the laser direction by the light intensity distribution from each small eye array. The simulation results carried out by MATLAB show that the model is suitable for fast aiming and tracking laser channel of space nodes with little location accuracy error range.

The evaluation of ventricular function is important for the diagnosis of cardiovascular diseases. It typically involves measurement of the left ventricular (LV) mass and LV cavity volume. Manual delineation of the myocardial contours is time-consuming and dependent on the subjective experience of the expert observer. In this paper, a multi-atlas method is proposed for cardiac magnetic resonance (MR) image segmentation. The proposed method is novel in two aspects. First, it formulates a patch-based label fusion model in a Bayesian framework. Second, it improves image registration accuracy by utilizing label information, which leads to improvement of segmentation accuracy. The proposed method was evaluated on a cardiac MR image set of 28 subjects. The average Dice overlap metric of our segmentation is 0.92 for the LV cavity, 0.89 for the right ventricular cavity and 0.82 for the myocardium. The results show that the proposed method is able to provide accurate information for clinical diagnosis.

Despite the breakthroughs in accuracy and speed of single image super-resolution using faster and deeper convolutional neural networks, one central problem remains largely unsolved: how do we recover the finer texture details when we super-resolve at large upscaling factors? The behavior of optimization-based super-resolution methods is principally driven by the choice of the objective function. Recent work has largely focused on minimizing the mean squared reconstruction error. The resulting estimates have high peak signal-to-noise ratios, but they are often lacking high-frequency details and are perceptually unsatisfying in the sense that they fail to match the fidelity expected at the higher resolution. In this paper, we present SRGAN, a generative adversarial network (GAN) for image super-resolution (SR). To our knowledge, it is the first framework capable of inferring photo-realistic natural images for 4x upscaling factors. To achieve this, we propose a perceptual loss function which consists of an adversarial loss and a content loss. The adversarial loss pushes our solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images. In addition, we use a content loss motivated by perceptual similarity instead of similarity in pixel space. Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks. An extensive mean-opinion-score (MOS) test shows hugely significant gains in perceptual quality using SRGAN. The MOS scores obtained with SRGAN are closer to those of the original high-resolution images than to those obtained with any state-of-the-art method., 19 pages, 15 figures, 2 tables, accepted for oral presentation at CVPR, main paper + some supplementary material

Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods., CVPR 2016 paper with updated affiliations and supplemental material, fixed typo in equation 4

We consider the problem of face swapping in images, where an input identity is transformed into a target identity while preserving pose, facial expression and lighting. To perform this mapping, we use convolutional neural networks trained to capture the appearance of the target identity from an unstructured collection of his/her photographs. This approach is enabled by framing the face swapping problem in terms of style transfer, where the goal is to render an image in the style of another one. Building on recent advances in this area, we devise a new loss function that enables the network to produce highly photorealistic results. By combining neural networks with simple pre- and post-processing steps, we aim at making face swap work in real-time with no input from the user.

We propose a novel technique for the localisation of Left Ventricular LV scar based on local motion descriptors. Cardiac MR imaging is employed to construct a spatio-temporal motion atlas where the LV motion of different subjects can be directly compared. Local motion descriptors are derived from the motion atlas and dictionary learning is used for scar classification. Preliminary results on a cohort of 20 patients show a sensitivity and specificity of $$80\,\%$$ and $$87\,\%$$ in a binary classification setting.

Extracting centerlines of coronary arteries is challenging but important in clinical applications of cardiac computed tomography angiography (CTA). Since manual annotation of coronary arteries is time-consuming, labor-intensive and subject to intra- and inter-variations, we propose a new method to fully automatically extract the coronary centerlines. We first develop a new image filter which generates pixels with salient vessel features within a given window. This filter hence can capture sparsely distributed but important vessel points, enabling the minimal path (MP) process to track the key centerline points at different resolution of the images. Then, we reformulate the filter for multi-resolution fast marching, which not only can speed up the coronary tracking process, but also can help the front propagation to step over the indistinct segments of the coronary artery such as at the locations of stenosis. We embed this scheme into the MP framework to develop a multi-resolution multi-model approach (MMP), where the extracted centerlines from low-resolution MP serve as prior and constraints for the high-resolution process. We evaluated the performance of this method using the Rotterdam CTA training data and the coronary artery algorithm evaluation framework. The average inside of our extraction was 0.51 mm and the overlap was 72.9 %. The mean runtime on the original resolution CTA images was 3.4 min using the MMP method.

Biomedical research, combining multi-modal image and geometry data, presents unique challenges for data visualization, processing, and quantitative analysis. Medical imaging provides rich information, from anatomical to deformation, but extracting this to a coherent picture across image modalities with preserved quality is not trivial. Addressing these challenges and integrating visualization with image and quantitative analysis results in Eidolon, a platform which can adapt to rapidly changing research workflows. In this paper we outline Eidolon, a software environment aimed at addressing these challenges, and discuss the novel integration of visualization and analysis components. These capabilities are demonstrated through the example of cardiac strain analysis, showing the Eidolon supports and enhances the workflow.

In this paper, we present a novel technique based on nonrigid image registration for myocardial motion estimation using both untagged and 3-D tagged MR images. The novel aspect of our technique is its simultaneous usage of complementary information from both untagged and 3-D tagged MR images. To estimate the motion within the myocardium, we register a sequence of tagged and untagged MR images during the cardiac cycle to a set of reference tagged and untagged MR images at end-diastole. The similarity measure is spatially weighted to maximize the utility of information from both images. In addition, the proposed approach integrates a valve plane tracker and adaptive incompressibility into the framework. We have evaluated the proposed approach on 12 subjects. Our results show a clear improvement in terms of accuracy compared to approaches that use either 3-D tagged or untagged MR image information alone. The relative error compared to manually tracked landmarks is less than 15% throughout the cardiac cycle. Finally, we demonstrate the automatic analysis of cardiac function from the myocardial deformation fields.

Background: Right ventricular (RV) function has prognostic value in disease cohorts and the general population although its complex geometry and motion make rapid assessment difficult. Several two-dimensional regional measures are used to estimate global RV function, however the optimal measure is not known. Methods: High-resolution 3-dimensional cardiac magnetic resonance (3D-CMR) cine imaging of the whole heart was acquired in 300 healthy volunteers and a computational model of RV motion created. From this model, points where regional function was significantly associated with global function were identified, and a two-dimensional, Optimized Single-Point Marker (SPM-O) of global function developed. This marker was prospectively compared with tricuspid annular plane systolic excursion (TAPSE), septum-freewall displacement (SFD) and their fractional derivatives (TAPSE-F, SFD-F) in a test cohort (n=300). Performance in predicting RVEF was compared by correlation, bootstrapped linear regression and mean absolute error. Results: Three-dimensional imaging suggested that RVEF was significantly associated with systolic function in a contiguous 7.3cm 2 patch of the basal RV freewall which combined components of transverse (38%), longitudinal (35%) and circumferential (27%) contraction. SPM-O was identified where this patch coincided with the four-chamber view and combined longitudinal and transverse contraction. In the test cohort, all single-point surrogates correlated with RVEF (p Conclusions: Systolic motion of the basal RV freewall in the four-chamber plane is a more accurate surrogate of global function than other two-dimensional estimators. However, the performance of two-dimensional estimators is fundamentally limited by a failure to adequately summarize three-dimensional contractile patterns, such as those found in the right ventricle.

Background The prevalence of hypertension has doubled over the previous decade and represents a significant public health problem worldwide. Excess cardiovascular risk due to increased blood pressure shows no evidence of a threshold to at least 115/75 mmHg. Left ventricular (LV) hypertrophy has a strong association with essential hypertension and is an independent risk factor for mortality. However, there is little data on whether systolic blood pressure (SBP) influences LV morphology and function in a healthy population. Furthermore it is not clear if any phenotypical changes are part of an adaptive response to increasing afterload or the beginning of a maladaptive process that ultimately leads to cardiomyopathy.

both age (b= 0.08; P

Cardiac MR imaging contains rich information that can be used to investigate the anatomy and function of the heart. In this paper, we demonstrate that it is possible to learn anatomical and functional information from cardiac MR imaging without explicit segmentation in order to predict clinical variables such as blood pressure with high accuracy. To learn the anatomical variations, we build manifolds of different time points across different subjects. In addition, we investigate two different approaches to incorporate motion information into a manifold, and compare these manifolds to a manifold learned from a single time point. Combining both inter- and intra-subject variation, we are able to construct accurate and reliable classifiers to predict clinical variables. Our proposed method does not require any explicit image segmentation and motion estimation and is able to predict clinical variables with good accuracy.

Motion, together with shape, reflect important aspects of cardiac function. In this work, a new method is proposed for learning of a cardiac motion descriptor from a data-driven perspective. The resulting descriptor can characterise the global motion pattern of the left ventricle with a much lower dimension than the original motion data. It has demonstrated its predictive power on two exemplar classification tasks on a large cohort of 1093 normal subjects.

Automated left ventricle (LV) segmentation in 3D ultrasound (3D-US) remains a challenging research problem due to variable image quality and limited field-of-view. Modern segmentation approaches (shape, appearance and contour model based surface fitting) require an accurate initialization and good image boundary features to obtain reliable and consistent results. They are therefore not well suited for this problem. The proposed method overcomes those limitations with a novel and generic 3D-US image boundary representation technique: Probabilistic Edge Map (PEM). This new representation captures regularized and complete edge responses from standard 3D-US images. PEM is utilized in a multi-atlas LV segmentation framework to spatially align target and atlas images. Experiments on data from the MICCAI CETUS challenge show that the proposed approach is better suited for LV segmentation than the active contour, appearance and voxel classification approaches, achieving lower surface distance errors and better LV volume estimates.

Myocardial stiffness is a clinical biomarker used to diagnose and stratify diseases such as heart failure. This biomechanical property can be inferred from the personalisation of computational cardiac models to clinical measures. Nevertheless, previous attempts have been unable to determine a unique set of material constitutive parameters. In this study we address this shortcoming by proposing a new cost function that allows us to uncouple key parameters and uniquely describe passive material properties in patients from available clinical data.

Atlases encode valuable anatomical and functional information from a population. In this work, a bi-ventricular cardiac atlas was built from a unique data set, which consists of high resolution cardiac MR images of 1000+ normal subjects. Based on the atlas, statistical methods were used to study the variation of cardiac shapes and the distribution of cardiac motion across the spatio-temporal domain. We have shown how statistical parametric mapping (SPM) can be combined with a general linear model to study the impact of gender and age on regional myocardial wall thickness. Finally, we have also investigated the influence of the population size on atlas construction and atlas-based analysis. The high resolution atlas, the statistical models and the SPM method will benefit more studies on cardiac anatomy and function analysis in the future.

Multi-atlas segmentation infers the target image segmentation by combining prior anatomical knowledge encoded in multiple atlases. It has been quite successfully applied to medical image segmentation in the recent years, resulting in highly accurate and robust segmentation for many anatomical structures. However, to guide the label fusion process, most existing multi-atlas segmentation methods only utilise the intensity information within a small patch during the label fusion process and may neglect other useful information such as gradient and contextual information (the appearance of surrounding regions). This paper proposes to combine the intensity, gradient and contextual information into an augmented feature vector and incorporate it into multi-atlas segmentation. Also, it explores the alternative to the K nearest neighbour (KNN) classifier in performing multi-atlas label fusion, by using the support vector machine (SVM) for label fusion instead. Experimental results on a short-axis cardiac MR data set of 83 subjects have demonstrated that the accuracy of multi-atlas segmentation can be significantly improved by using the augmented feature vector. The mean Dice metric of the proposed segmentation framework is 0.81 for the left ventricular myocardium on this data set, compared to 0.79 given by the conventional multi-atlas patch-based segmentation (Coupé et al., 2011; Rousseau et al., 2011). A major contribution of this paper is that it demonstrates that the performance of non-local patch-based segmentation can be improved by using augmented features.

The quantification of similarity between image segmentations is a complex yet important task. The ideal similarity measure should be unbiased to segmentations of different volume and complexity, and be able to quantify and visualise segmentation bias. Similarity measures based on overlap, e.g. Dice score, or surface distances, e.g. Hausdorff distance, clearly do not satisfy all of these properties. To address this problem, we introduce Patch-based Evaluation of Image Segmentation (PEIS), a general method to assess segmentation quality. Our method is based on finding patch correspondences and the associated patch displacements, which allow the estimation of segmentation bias. We quantify both the agreement of the segmentation boundary and the conservation of the segmentation shape. We further assess the segmentation complexity within patches to weight the contribution of local segmentation similarity to the global score. We evaluate PEIS on both synthetic data and two medical imaging datasets. On synthetic segmentations of different shapes, we provide evidence that PEIS, in comparison to the Dice score, produces more comparable scores, has increased sensitivity and estimates segmentation bias accurately. On cardiac magnetic resonance (MR) images, we demonstrate that PEIS can evaluate the performance of a segmentation method independent of the size or complexity of the segmentation under consideration. On brain MR images, we compare five different automatic hippocampus segmentation techniques using PEIS. Finally, we visualise the segmentation bias on a selection of the cases.

We propose a consistent approach to automatically segmenting longitudinal magnetic resonance scans of pathological brains. Using symmetric intra-subject registration, we align corresponding scans. In an expectation-maximization framework we exploit the availability of probabilistic segmentation estimates to perform a symmetric intensity normalisation. We introduce a novel technique to perform symmetric differential bias correction for images in presence of pathologies. To achieve a consistent multi-time-point segmentation, we propose a patch-based coupling term using a spatially and temporally varying Markov random field. We demonstrate the superior consistency of our method by segmenting repeat scans into 134 regions. Furthermore, the approach has been applied to segment baseline and six month follow-up scans from 56 patients who have sustained traumatic brain injury (TBI). We find significant correlations between regional atrophy rates and clinical outcome: Patients with poor outcome showed a much higher thalamic atrophy rate (4.9 ± 3.4%) than patients with favourable outcome (0.6 ± 1.9%).

International audience; Magnetic Resonance Imaging (MRI), a reference examination for cardiac morphology and function in humans, allows to image the cardiac right ventricle (RV) with high spatial resolution. The segmentation of the RV is a difficult task due to the variable shape of the RV and its ill-defined borders in these images. The aim of this paper is to evaluate several RV segmentation algorithms on common data. More precisely, we report here the results of the Right Ventricle Segmentation Challenge (RVSC), concretized during the MICCAI'12 Conference with an on-site competition. Seven automated and semi-automated methods have been considered, along them three atlas-based methods, two prior based methods, and two prior-free, image-driven methods that make use of cardiac motion. The obtained contours were compared against a manual tracing by an expert cardiac radiologist, taken as a reference, using Dice metric and Hausdorff distance. We herein describe the cardiac data composed of 48 patients, the evaluation protocol and the results. Best results show that an average 80% Dice accuracy and a 1cm Hausdorff distance can be expected from semi-automated algorithms for this challenging task on the datasets, and that an automated algorithm can reach similar performance, at the expense of a high computational burden. Data are now publicly available and the website remains open for new submissions (http://www.litislab.eu/rvsc/).

Background Cardiac phenotypes, such as left ventricular (LV) mass, demonstrate high heritability although most genes associated with these complex traits remain unidentified. Genome-wide association studies (GWAS) have relied on conventional 2D cardiovascular magnetic resonance (CMR) as the gold-standard for phenotyping. However this technique is insensitive to the regional variations in wall thickness which are often associated with left ventricular hypertrophy and require large cohorts to reach significance. Here we test whether automated cardiac phenotyping using high spatial resolution CMR atlases can achieve improved precision for mapping wall thickness in healthy populations and whether smaller sample sizes are required compared to conventional methods. Methods LV short-axis cine images were acquired in 138 healthy volunteers using standard 2D imaging and 3D high spatial resolution CMR. A multi-atlas technique was used to segment and co-register each image. The agreement between methods for end-diastolic volume and mass was made using Bland-Altman analysis in 20 subjects. The 3D and 2D segmentations of the LV were compared to manual labeling by the proportion of concordant voxels (Dice coefficient) and the distances separating corresponding points. Parametric and nonparametric data were analysed with paired t-tests and Wilcoxon signed-rank test respectively. Voxelwise power calculations used the interstudy variances of wall thickness. Results The 3D volumetric measurements showed no bias compared to 2D imaging. The segmented 3D images were more accurate than 2D images for defining the epicardium (Dice: 0.95 vs 0.93, P

The automatic segmentation of cardiac magnetic resonance images poses many challenges arising from the large variation between different anatomies, scanners and acquisition protocols. In this paper, we address these challenges with a global graph search method and a novel spectral embedding of the images. Firstly, we propose the use of an approximate graph search approach to initialize patch correspondences between the image to be segmented and a database of labelled atlases. Then, we propose an innovative spectral embedding using a multi-layered graph of the images in order to capture global shape properties. Finally, we estimate the patch correspondences based on a joint spectral representation of the image and atlases. We evaluated the proposed approach using 155 images from the recent MICCAI SATA segmentation challenge and demonstrated that the proposed algorithm significantly outperforms current state-of-the-art methods on both training and test sets.

Background Late Gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging can be used to visualise regions of fibrosis and scarring in the left atrium (LA) myocardium. This can be important for treatment stratification of patients with atrial fibrillation (AF) and for assessment of treatment after radio frequency catheter ablation (RFCA). In this paper we present a standardised evaluation benchmarking framework for algorithms segmenting fibrosis and scar from LGE CMR images. The algorithms reported are the response to an open challenge that was put to the medical imaging community through an ISBI (IEEE International Symposium on Biomedical Imaging) workshop. Methods The image database consisted of 60 multicenter, multivendor LGE CMR image datasets from patients with AF, with 30 images taken before and 30 after RFCA for the treatment of AF. A reference standard for scar and fibrosis was established by merging manual segmentations from three observers. Furthermore, scar was also quantified using 2, 3 and 4 standard deviations (SD) and full-width-at-half-maximum (FWHM) methods. Seven institutions responded to the challenge: Imperial College (IC), Mevis Fraunhofer (MV), Sunnybrook Health Sciences (SY), Harvard/Boston University (HB), Yale School of Medicine (YL), King’s College London (KCL) and Utah CARMA (UTA, UTB). There were 8 different algorithms evaluated in this study. Results Some algorithms were able to perform significantly better than SD and FWHM methods in both pre- and post-ablation imaging. Segmentation in pre-ablation images was challenging and good correlation with the reference standard was found in post-ablation images. Overlap scores (out of 100) with the reference standard were as follows: Pre: IC = 37, MV = 22, SY = 17, YL = 48, KCL = 30, UTA = 42, UTB = 45; Post: IC = 76, MV = 85, SY = 73, HB = 76, YL = 84, KCL = 78, UTA = 78, UTB = 72. Conclusions The study concludes that currently no algorithm is deemed clearly better than others. There is scope for further algorithmic developments in LA fibrosis and scar quantification from LGE CMR images. Benchmarking of future scar segmentation algorithms is thus important. The proposed benchmarking framework is made available as open-source and new participants can evaluate their algorithms via a web-based interface.

FFD represent a widely used model for the non-rigid registration of medical images. The balance between robustness to noise and accuracy in modelling localised motion is typically controlled by the control point grid spacing and the amount of regularisation. More recently, TFFD have been proposed which extend the FFD approach in order to recover smooth motion from temporal image sequences. In this paper, we revisit the classic FFD approach and propose a sparse representation using the principles of compressed sensing. The sparse representation can model both global and local motion accurately and robustly. We view the registration as a deformation reconstruction problem. The deformation is reconstructed from a pair of images (or image sequences) with a sparsity constraint applied to the parametric space. Specifically, we introduce sparsity into the deformation via L1 regularisation, and apply a bending energy regularisation between neighbouring control points within each level to encourage a grouped sparse solution. We further extend the sparsity constraint to the temporal domain and propose a TSFFD which can capture fine local details such as motion discontinuities in both space and time without sacrificing robustness. We demonstrate the capabilities of the proposed framework to accurately estimate deformations in dynamic 2D and 3D image sequences. Compared to the classic FFD and TFFD approach, a significant increase in registration accuracy can be observed in natural images as well as in cardiac images.

Neointima thickening plays a decisive role in coronary restenosis after stenting. The aim of this study is to detect neointima tissue in intravascular optical coherence tomography (IVOCT) sequences. We developed a multi-atlas based segmentation method to detect neointima without stent struts locations. The atlases are selected by measurements of stenosis and a similarity metric. The probability map is then used to estimate neointima label in the unseen image. To account for the registration errors, a patch-based label fusion approach is applied. Validation is performed using 18 typical in-vivo IVOCT sequences. The comparison against manual expert segmentation and other fusion approaches demonstrates that the proposed neointima identification is robust and accurate.

In this work, we investigate the computation on a shape manifold for atlas generation and application to atlas propagation and segmentation. We formulate the computation of Frechet mean via the constant velocity fields and Log-Euclidean framework for Nadaraya-Watson kernel regression modeling. In this formulation, we directly compute the Frechet mean of shapes via fast vectorial operations on the velocity fields. By using image similarity metric to estimate the distance of shapes in the assumed manifold, we can estimate a close shape of an unseen image using Naderaya-Watson kernel regression function. We applied this estimation to generate subject-specific atlases for whole heart segmentation of MRI data. The segmentation results on clinical data demonstrated an improved performance compared to existing methods, thanks to the usage of subject-specific atlases which had more similar shapes to the unseen images.

Increasing attention has been focused on the estimation of the deformation of the endocardium to aid the diagnosis of cardiac malfunction. Landmark tracking can provide sparse, anatomically relevant constraints to help establish correspondences between images being tracked or registered. However, landmarks on the endocardium are often characterized by ambiguous appearance in cardiac MR images which makes the extraction and tracking of these landmarks problematic. In this paper we propose an automatic framework to select and track a sparse set of distinctive landmarks in the presence of relatively large deformations in order to capture the endocardial motion in cardiac MR sequences. To achieve this a sparse set of the landmarks is identified using an entropy-based approach. In particular we use singular value decomposition (SVD) to reduce the search space and localize the landmarks with relatively large deformation across the cardiac cycle. The tracking of the sparse set of landmarks is performed simultaneously by optimizing a two-stage Markov Random Field (MRF) model. The tracking result is further used to initialize registration based dense motion tracking. We have applied this framework to extract a set of landmarks at the endocardial border of the left ventricle in MR image sequences from 51 subjects. Although the left ventricle undergoes a number of different deformations, we show how the radial, longitudinal motion and twisting of the endocardial surface can be captured by the proposed approach. Our experiments demonstrate that motion tracking using sparse landmarks can outperform conventional motion tracking by a substantial amount, with improvements in terms of tracking accuracy of 20:8% and 19:4% respectively.

In this paper we present a benchmarking framework for the validation of cardiac motion analysis algorithms. The reported methods are the response to an open challenge that was issued to the medical imaging community through a MICCAI workshop. The database included magnetic resonance (MR) and 3D ultrasound (3DUS) datasets from a dynamic phantom and 15 healthy volunteers. Participants processed 3D tagged MR datasets (3DTAG), cine steady state free precession MR datasets (SSFP) and 3DUS datasets, amounting to 1158 image volumes. Ground-truth for motion tracking was based on 12 landmarks (4 walls at 3 ventricular levels). They were manually tracked by two observers in the 3DTAG data over the whole cardiac cycle, using an in-house application with 4D visualization capabilities. The median of the inter-observer variability was computed for the phantom dataset (0.77 mm) and for the volunteer datasets (0.84 mm). The ground-truth was registered to 3DUS coordinates using a point based similarity transform. Four institutions responded to the challenge by providing motion estimates for the data: Fraunhofer MEVIS (MEVIS), Bremen, Germany; Imperial College London - University College London (IUCL), UK; Universitat Pompeu Fabra (UPF), Barcelona, Spain; Inria-Asclepios project (INRIA), France. Details on the implementation and evaluation of the four methodologies are presented in this manuscript. The manually tracked landmarks were used to evaluate tracking accuracy of all methodologies. For 3DTAG, median values were computed over all time frames for the phantom dataset (MEVIS = 1.20 mm, IUCL = 0.73 mm, UPF = 1.10 mm, INRIA = 1.09 mm) and for the volunteer datasets (MEVIS = 1.33 mm, IUCL = 1.52 mm, UPF = 1.09 mm, INRIA = 1.32 mm). For 3DUS, median values were computed at end diastole and end systole for the phantom dataset (MEVIS = 4.40 mm, UPF = 3.48 mm, INRIA = 4.78 mm) and for the volunteer datasets (MEVIS = 3.51 mm, UPF = 3.71 mm, INRIA = 4.07 mm). For SSFP, median values were computed at end diastole and end systole for the phantom dataset(UPF = 6.18 mm, INRIA = 3.93 mm) and for the volunteer datasets (UPF = 3.09 mm, INRIA = 4.78 mm). Finally, strain curves were generated and qualitatively compared. Good agreement was found between the different modalities and methodologies, except for radial strain that showed a high variability in cases of lower image quality.

The accurate measurement of 3D cardiac function is an important task in the analysis of cardiac magnetic resonance (MR) images. However, short-axis image acquisitions with thick slices are commonly used in clinical practice due to constraints of acquisition time, signal-to-noise ratio and patient compliance. In this situation, the estimation of a high-resolution image can provide an approximation of the underlaying 3D measurements. In this paper, we develop a novel algorithm for the estimation of high-resolution cardiac MR images from single short-axis cardiac MR image stacks. First, we propose to use a novel approximate global search approach to find patch correspondence between the short-axis MR image and a set of atlases. Then, we propose an innovative super-resolution model which does not require explicit motion estimation. Finally, we build an expectation-maximization framework to optimize the model. We validate the proposed approach using images from 19 subjects with 200 atlases and show that the proposed algorithm significantly outperforms conventional interpolation such as linear or B-spline interpolation. In addition, we show that the super-resolved images can be used for the reproducible estimation of 3D cardiac functional indices.

Accelerated methods for acquiring phase contrast (PC) MRI allow the acquisition of 4D flow data of the whole heart in clinically acceptable times. These datasets are becoming interesting both for clinicians --- to better stratify diagnosis --- and in the modeling community --- to constrain patient-specific models. One of the difficulties related to PC data is a limited accuracy in the regions of low flow such as close to the myocardial wall, where the velocity field may even produce observed blood motion across the endocardial surface. To address this issue we propose to constrain the motion of blood in cavity during the analysis by using cine MRI and replacing the PC velocity in the peri-myocardial zone by neighboring tissue velocity obtained by analysis of 3D tagged MRI. We demonstrate the effect of these corrections on 2 healthy volunteer datasets and on one patient with a hypoplastic left ventricle.

Background Survival in pulmonary hypertension depends on right ventricular adaptation to pressure overload, but the three-dimensional (3D) patterns of function that are most discriminative of subsequent outcome are not known. The aim of this study was to compare the prognostic accuracy of artificial intelligence combined with 3D models of the right ventricle against conventional prognostic markers in predicting survival in patients with pulmonary hypertension. Methods Patients with suspected pulmonary hypertension were prospectively enrolled and underwent cardiac MRI, right heart catheterisation, and 6-min walk testing, and received standard medical care and follow-up. From cardiac MRI, 3D right ventricular function models were developed and the patterns of function predictive of survival learnt by artificial intelligence. All-cause mortality prediction was compared between artificial intelligence and standard clinical markers using Cox proportional hazards. Findings 256 patients had confirmed pulmonary hypertension, investigations within 3 months, and follow-up for a median of 4·0 years [IQR 2·0–5·7]. Artificial intelligence identified specific regional and directional patterns of function, which predicted survival and was a stronger predictor (median area under the curve 0·72, IQR 0·69–0·76; p Interpretation The right ventricle in pulmonary hypertension demonstrates prognostic regional and directional adaptations that can be revealed by artificial intelligence methods, thereby improving risk-stratification and mechanistic insights. The variety of pulmonary hypertension subtypes and treatment regimens included might limit applicability to subgroups, but demonstrates efficacy across a spectrum of disease and treatment. Artificial intelligence methods could have important additional benefit in analysing complex cardiovascular datasets. Funding Medical Research Council UK, National Institute for Health Research Biomedical Research Centre, Imperial College Healthcare NHS Trust and Imperial College London, British Heart Foundation (project grant PG/12/27/29489, special grant SP/10/10/28431), Wellcome Trust GlaxoSmithKline Fellowship Grant.

Background Progressive heart failure due to remodeling is a major cause of morbidity and mortality following myocardial infarction. Conventional clinical imaging measures global volume changes, and currently there is no means of assessing regional myocardial dilatation in relation to ischemic burden. Here we use 3D co-registration of Cardiovascular Magnetic Resonance (CMR) images to assess the long-term effects of ischemia-reperfusion injury on left ventricular structure after acute ST-elevation myocardial infarction (STEMI). Methods Forty six patients (age range 33–77 years) underwent CMR imaging within 7 days following primary percutaneous coronary intervention (PPCI) for acute STEMI with follow-up at one year. Functional cine imaging and Late Gadolinium Enhancement (LGE) were segmented and co-registered. Local left ventricular wall dilatation was assessed by using intensity-based similarities to track the structural changes in the heart between baseline and follow-up. Results are expressed as means, standard errors and 95% confidence interval (CI) of the difference. Results Local left ventricular remodeling within infarcted myocardium was greater than in non-infarcted myocardium (1.6% ± 1.0 vs 0.3% ± 0.9, 95% CI: -2.4% – -0.2%, P = 0.02). One-way ANOVA revealed that transmural infarct thickness had a significant effect on the degree of local remodeling at one year (P 20% (4.8% ± 1.4 vs −0.15% ± 1.2, 95% CI: -8.9% – -0.9%, P = 0.017). Conclusions The severity of ischemic injury has a significant effect on local ventricular wall remodeling with only modest dilatation observed within non-ischemic myocardium. Limitation of chronic remodeling may therefore depend on therapies directed at modulating ischemia-reperfusion injury. CMR co-registration has potential for assessing dynamic changes in ventricular structure in relation to therapeutic interventions.

Optical coherence tomography (OCT) is a light-based, high resolution imaging technique to guide stent deployment procedure for stenosis. OCT can accurately differentiate the most superficial layers of the vessel wall as well as stent struts and the vascular tissue surrounding them. In this paper, we automatically detect the struts of coronary stents present in OCT sequences. We propose a novel method to detect the strut shadow zone and accurately segment and reconstruct the strut in 3D. The estimation of the position of the strut shadow zone is the key requirement which enables the strut segmentation. After identification of the shadow zone we use probability map to estimate stent strut positions. This method can be applied to cross-sectional OCT images to detect the struts. Validation is performed using simulated data as well as in four in-vivo OCT sequences and the accuracy of strut detection is over 90%. The comparison against manual expert segmentation demonstrates that the proposed strut identification is robust and accurate.

We present a fully automatic framework for cardiac motion tracking based on non-rigid image registration for the analysis of myocardial motion using both untagged and 3D tagged MR images. We detect and track anatomical landmarks in the heart and combine this with intensity-based motion tracking to allow accurately model cardiac motion while significantly reduce the computational complexity. A collaborative similarity measure simultaneously computed in three LA views is employed to register a sequence of images taken during the cardiac cycle to a reference image taken at end-diastole. We then integrate a valve plane tracker into the framework which uses short-axis and long-axis untagged MR images as well as 3D tagged images to estimate a fully four-dimensional motion field of the left ventricle.

Registration and segmentation uncertainty may be important information to convey to a user when automatic image analysis is performed. Uncertainty information may be used to provide additional diagnostic information to traditional analysis of cardiac function. In this paper, we develop a framework for the automatic segmentation of the cardiac anatomy from multiple MR images. We also define the registration and segmentation uncertainty and explore its use for diagnostic purposes. Our framework uses cardiac MR image sequences that are widely available in clinical practice. We improve the performance of the cardiac segmentation algorithms by combining information from multiple MR images using a graph-cut based segmentation. We evaluate this framework on images from 32 subjects: 13 patients with ischemic cardiomyopathy, 14 patients with dilated cardiomyopathy and 5 normal volunteers. Our results indicate that the proposed method is capable of producing segmentation results with very high robustness and high accuracy with minimal user interaction across all subject groups. We also show that registration and segmentation uncertainties are good indicators for segmentation failures as well as good predictors for the functional abnormality of the subject.

Non-rigid image registration using free-form deformations (FFD) is a widely used technique in medical image registration. The balance between robustness and accuracy is controlled by the control point grid spacing and the amount of regularization. In this paper, we revisit the classic FFD registration approach and propose a sparse representation for FFDs using the principles of compressed sensing. The sparse free-form deformation model (SFFD) can capture fine local details such as motion discontinuities without sacrificing robustness. We demonstrate the capabilities of the proposed framework to accurately estimate smooth as well as discontinuous deformations in 2D and 3D image sequences. Compared to the classic FFD approach, a significant increase in registration accuracy can be observed in natural images (61%) as well as in cardiac MR images (53%) with discontinuous motions.

In this paper, we present a novel software tool designed and implemented to simplify the development process of Multimodal Human-Computer Interaction (MHCI) systems. This tool, which is called the HCI⁁2 Workbench, exploits a Publish / Subscribe (P/S) architecture [13] [14] to facilitate efficient and reliable inter-module data communication and runtime system management. In addition, through a combination of SDK, software tools, and standardized description / configuration file semantics, the HCI⁁2 Workbench provides an easy-to-follow procedure for developing highly flexible and reusable modules. Moreover, the HCI⁁2 Workbench features system persistence and portability by using standardized module packaging method and system configuration files. Last but not least, usability was another major concern. Unlike other similar tool, including Psyclone [9] and ActiveMQ [10], the HCI⁁2 Workbench provides a complete graphical environment to support every step in a typical MHCI system development process, including module program development and debugging, module packaging, module management, system configuration, module and system testing, in a convenient and intuitive manner. To help demonstrating the HCI⁁2 Workbench, we also present a readily-applicable system developed using our tool. This open-source demo system, which is called the CamGame, consists of an interactive system allowing users to play a computer game using hand-held marker(s) and low-cost camera(s) instead of keyboard and mouse.