Search

Qiu et al. present a novel deep learning strategy to generate high-resolution visualizations of Alzheimer’s disease risk in humans that are highly interpretable and can accurately predict Alzheimer’s disease status. They then test the model by comparing its performance to that of neurologists and neuropathological data., Alzheimer’s disease is the primary cause of dementia worldwide, with an increasing morbidity burden that may outstrip diagnosis and management capacity as the population ages. Current methods integrate patient history, neuropsychological testing and MRI to identify likely cases, yet effective practices remain variably applied and lacking in sensitivity and specificity. Here we report an interpretable deep learning strategy that delineates unique Alzheimer’s disease signatures from multimodal inputs of MRI, age, gender, and Mini-Mental State Examination score. Our framework linked a fully convolutional network, which constructs high resolution maps of disease probability from local brain structure to a multilayer perceptron and generates precise, intuitive visualization of individual Alzheimer’s disease risk en route to accurate diagnosis. The model was trained using clinically diagnosed Alzheimer’s disease and cognitively normal subjects from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset (n = 417) and validated on three independent cohorts: the Australian Imaging, Biomarker and Lifestyle Flagship Study of Ageing (AIBL) (n = 382), the Framingham Heart Study (n = 102), and the National Alzheimer’s Coordinating Center (NACC) (n = 582). Performance of the model that used the multimodal inputs was consistent across datasets, with mean area under curve values of 0.996, 0.974, 0.876 and 0.954 for the ADNI study, AIBL, Framingham Heart Study and NACC datasets, respectively. Moreover, our approach exceeded the diagnostic performance of a multi-institutional team of practicing neurologists (n = 11), and high-risk cerebral regions predicted by the model closely tracked post-mortem histopathological findings. This framework provides a clinically adaptable strategy for using routinely available imaging techniques such as MRI to generate nuanced neuroimaging signatures for Alzheimer’s disease diagnosis, as well as a generalizable approach for linking deep learning to pathophysiological processes in human disease.

Quantifying heterogeneity in Alzheimer’s disease (AD) risk is critical for individualized care and management. Recent attempts to assess AD heterogeneity have used structural (magnetic resonance imaging (MRI)-based) or functional (Aβ or tau) imaging, which focused on generating quartets of atrophy patterns and protein spreading, respectively. Here we present a computational framework that facilitated the identification of subtypes based on their risk of progression to AD. We used cerebrospinal fluid (CSF) measures of Aβ from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (n=544, discovery cohort) as well as the National Alzheimer’s Coordinating Center (NACC) (n=508, validation cohort), and risk-stratified individuals with mild cognitive impairment (MCI) into quartiles (high-risk (H), intermediate-high risk (IH), intermediate-low risk (IL), and low-risk (L)). Patients were divided into subgroups utilizing patterns of brain atrophy found in each of these risk-stratified quartiles. We found H subjects to have a greater risk of AD progression compared to the other subtypes at 2- and 4-years in both the discovery and validation cohorts (ADNI: H subtype versus all others, p < 0.05 at 2 and 4 years; NACC: H vs. IL and LR at 2 years, p < 0.05, and a trend toward higher risk vs. IH, and p < 0.05 vs. IH, and L risk groups at 48 months with a trend toward lower survival vs. IL). Using MRI-based neural models that fused various deep neural networks with survival analysis, we then predicted MCI to AD conversion. We used these models to identify subtype-specific regions that demonstrate the largest levels of atrophy-related importance, which had minimal overlap (Average pairwise Jaccard Similarity in regions between the top 5 subtypes, 0.25±0.05 (± std)). Neuropathologic changes characteristic of AD were present across all subtypes in comparable proportions (Chi-square test, p>0.05 for differences in ADNC, n=31). Our risk-based approach to subtyping individuals provides an objective means to intervene and tailor care management strategies at early stages of cognitive decline.

It remains difficult to characterize the source of pain in knee joints either using radiographs or magnetic resonance imaging (MRI). We sought to determine if advanced machine learning methods such as deep neural networks could distinguish knees with pain from those without it and identify the structural features that are associated with knee pain. We constructed a convolutional Siamese network to associate MRI scans obtained on subjects from the Osteoarthritis Initiative (OAI) with frequent unilateral knee pain comparing the knee with frequent pain to the contralateral knee without pain. The Siamese network architecture enabled pairwise learning of information from two-dimensional (2D) sagittal intermediate-weighted turbo spin echo slices obtained from similar locations on both knees. Class activation mapping (CAM) was utilized to create saliency maps, which highlighted the regions most associated with knee pain. The MRI scans and the CAMs of each subject were reviewed by an expert radiologist to identify the presence of abnormalities within the model-predicted regions of high association. Using 10-fold cross-validation, our model achieved an area under curve (AUC) value of 0.808. When individuals whose knee WOMAC pain scores were not discordant were excluded, model performance increased to 0.853. The radiologist review revealed that about 86% of the cases that were predicted correctly had effusion-synovitis within the regions that were most associated with pain. This study demonstrates a proof of principle that deep learning can be applied to assess knee pain from MRI scans. • Our article is the first to leverage a deep learning framework to associate MR images of the knee with knee pain. • We developed a convolutional Siamese network that had the ability to fuse information from multiple two-dimensional (2D) MRI slices from the knee with pain and the contralateral knee of the same individual without pain to predict unilateral knee pain. • Our model achieved an area under curve (AUC) value of 0.808. When individuals who had WOMAC pain scores that were not discordant for knees (pain discordance

Additional file 1: Figure S1. (A) Simple GAN architecture with a generator and a discriminator. (B) Simple FCN classification architecture for prediction of AD status. Figure S2. Number of cases from the ADNI cohort used for GAN and FCN model development. The cases covered by the black arrow indicate the ones used for the GAN model development and the cases covered by the red arrow indicate the ones used for the FCN model development. Figure S3. ANCOVA analysis to assess the mean image quality using SNR, BRISQUE and NIQE in the ADNI-training (a-c) and ADNI-validation (d-f) data. Table S1. Specific group differences in magnetic field strength (1.5T, 3T, 3T*) evaluated using Tukey's post hoc procedure across the image quality measures SNR, BRISQUE and NIQE in the ADNI-training, testing and validation data. Table S2. ANOVA analysis of AUCs from the ROC curves on the gender, age and scanner variables to demonstrate whether the AD classification performance differed on various groups. Figure S4. Performance of the FCN classifier based on the images generated using the simpleGAN architecture (Figure S1). (A) Sensitivity-specificity and (B) precision-recall curves are shown on the ADNI test, AIBL and NACC datasets, respectively. Figure S5. ANCOVA analysis to assess the mean image quality using SNR, BRISQUE and NIQE for simpleGAN model in the ADNI-test (a-c) and t-test results of NACC (d-f) and AIBL (g-i) data, respectively. Figure S6. ANCOVA analysis to assess the mean image quality using SNR, BRISQUE and NIQE for simpleGAN model in the ADNI-training (a-c) and ADNI-validation (d-f) data. The effect of independent variables such as age, education, MMSE scores, and type of scanner was evaluated using a stepwise forward selection process and models were adjusted accordingly. Table S3. Specific group differences in magnetic field strength (1.5T, 3T, 3T*) were evaluated using Tukey's post hoc procedure across the image quality measures SNR, BRISQUE and NIQE for the simpleGAN model in the ADNI-training, testing and validation data.

Background Generative adversarial networks (GAN) can produce images of improved quality but their ability to augment image-based classification is not fully explored. We evaluated if a modified GAN can learn from magnetic resonance imaging (MRI) scans of multiple magnetic field strengths to enhance Alzheimer’s disease (AD) classification performance. Methods T1-weighted brain MRI scans from 151 participants of the Alzheimer’s Disease Neuroimaging Initiative (ADNI), who underwent both 1.5-Tesla (1.5-T) and 3-Tesla imaging at the same time were selected to construct a GAN model. This model was trained along with a three-dimensional fully convolutional network (FCN) using the generated images (3T*) as inputs to predict AD status. Quality of the generated images was evaluated using signal to noise ratio (SNR), Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) and Natural Image Quality Evaluator (NIQE). Cases from the Australian Imaging, Biomarker & Lifestyle Flagship Study of Ageing (AIBL, n = 107) and the National Alzheimer’s Coordinating Center (NACC, n = 565) were used for model validation. Results The 3T*-based FCN classifier performed better than the FCN model trained using the 1.5-T scans. Specifically, the mean area under curve increased from 0.907 to 0.932, from 0.934 to 0.940, and from 0.870 to 0.907 on the ADNI test, AIBL, and NACC datasets, respectively. Additionally, we found that the mean quality of the generated (3T*) images was consistently higher than the 1.5-T images, as measured using SNR, BRISQUE, and NIQE on the validation datasets. Conclusion This study demonstrates a proof of principle that GAN frameworks can be constructed to augment AD classification performance and improve image quality.

BackgroundGenerative adversarial networks (GAN) can generate images of improved quality but their ability to augment image-based classification tasks is not fully explored.PurposeWe evaluated if a modified GAN can learn from MRI scans of multiple magnetic field strength to enhance Alzheimer’s disease (AD) classification performance.Materials and methodsT1-weighted brain MRI scans from 151 participants of the Alzheimer’s Disease Neuroimaging Initiative (ADNI), who underwent both 1.5 Tesla (1.5T) and 3 Tesla imaging at the same time were selected to construct a GAN model. This model was trained along with a three-dimensional fully convolutional network (FCN) using the generated images (1.5T*) as inputs to predict AD status. Quality of the generated images was evaluated using signal to noise ratio (SNR), Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) and Natural Image Quality Evaluator (NIQE). Data from the Australian Imaging, Biomarker & Lifestyle Flagship Study of Ageing (AIBL, n=107), and the National Alzheimer’s Coordinating Center (NACC, n=565) was used for model validation.ResultsThe mean quality of the generated (1.5T*) images was consistently higher than the 1.5T images, as measured using SNR, BRISQUE and NIQE on the validation datasets. The 1.5T*-based FCN classifier performed better than the FCN model trained using the 1.5T scans.Specifically, the mean area under curve increased from 0.907 to 0.932, from 0.934 to 0.940 and from 0.870 to 0.907 on the ADNI test, AIBL and NACC datasets, respectively.ConclusionThis study demonstrates that GAN frameworks can be constructed to simultaneously improve image quality and augment classification performance.Key pointsOur proposed generative adversarial network used 1.5 and 3 Tesla MR scans of the brain to generate images of improved quality, as estimated using no-reference image quality algorithms.Classification models of Alzheimer’s disease risk developed using the generated images had higher classification performance than the models developed using the original 1.5 Tesla scans.

Introduction Our aim was to investigate if the accuracy of diagnosing mild cognitive impairment (MCI) using the Mini–Mental State Examination (MMSE) and logical memory (LM) test could be enhanced by adding MRI data. Methods Data of individuals with normal cognition and MCI were obtained from the National Alzheimer Coordinating Center database (n = 386). Deep learning models trained on MRI slices were combined to generate a fused MRI model using different voting techniques to predict normal cognition versus MCI. Two multilayer perceptron (MLP) models were developed with MMSE and LM test results. Finally, the fused MRI model and the MLP models were combined using majority voting. Results The fusion model was superior to the individual models alone and achieved an overall accuracy of 90.9%. Discussion This study is a proof of principle that multimodal fusion of models developed using MRI scans, MMSE, and LM test data is feasible and can better predict MCI.

Alzheimer’s disease (AD) is the primary cause of dementia worldwide (1), with an increasing morbidity burden that may outstrip diagnosis and management capacity as the population ages. Current methods integrate patient history, neuropsychological testing and magnetic resonance imaging (MRI) to identify likely cases, yet effective practices remain variably-applied and lacking in sensitivity and specificity (2). Here we report an explainable deep learning strategy that delineates unique AD signatures from multimodal inputs of MRI, age, gender, and mini-mental state examination (MMSE) score. Our framework linked a fully convolutional network (FCN) to a multilayer perceptron (MLP) to construct high resolution maps of disease probability from local brain structure. This enabled precise, intuitive visualization of individual AD risk en route to accurate diagnosis. The model was trained using clinically-diagnosed AD and cognitively normal (NC) subjects from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset (n=417) (3), and validated on three independent cohorts: the Australian Imaging, Biomarker & Lifestyle Flagship Study of Ageing (AIBL, n=382) (4), the Framingham Heart Study (FHS, n=102) (5), and the National Alzheimer’s Coordinating Center (NACC, n=582) (6). Model performance was consistent across datasets, with mean accuracy values of 0.966, 0.948, 0.815, and 0.916 for ADNI, AIBL, FHS and NACC, respectively. Moreover, our approach exceeded the diagnostic performance of a multi-institutional team of practicing neurologists (n=11), and high-risk cerebral regions predicted by the model closely tracked postmortem histopathological findings. This framework provides a clinically-adaptable strategy for using routinely available imaging techniques such as MRI to generate nuanced neuroimaging signatures for AD diagnosis, as well as a generalizable approach for linking deep learning to pathophysiological processes in human disease.

Background and objectiveIt remains difficult to characterize pain in knee joints with osteoarthritis solely by radiographic findings. We sought to understand how advanced machine learning methods such as deep neural networks can be used to analyze raw MRI scans and predict bilateral knee pain, independent of other risk factors.MethodsWe developed a deep learning framework to associate information from MRI slices taken from the left and right knees of subjects from the Osteoarthritis Initiative with bilateral knee pain. Model training was performed by first extracting features from two-dimensional (2D) sagittal intermediate-weighted turbo spin echo slices. The extracted features from all the 2D slices were subsequently combined to directly associate using a fused deep neural network with the output of interest as a binary classification problem.ResultsThe deep learning model resulted in predicting bilateral knee pain on test data with 70.1% mean accuracy, 51.3% mean sensitivity, and 81.6% mean specificity. Systematic analysis of the predictions on the test data revealed that the model performance was consistent across subjects of different Kellgren-Lawrence grades.ConclusionThe study demonstrates a proof of principle that a machine learning approach can be applied to associate MR images with bilateral knee pain.SIGNIFICANCE AND INNOVATIONKnee pain is typically considered as an early indicator of osteoarthritis (OA) risk. Emerging evidence suggests that MRI changes are linked to pre-clinical OA, thus underscoring the need for building image-based models to predict knee pain. We leveraged a state-of-the-art machine learning approach to associate raw MR images with bilateral knee pain, independent of other risk factors.

In computer systems, information leaks from the physical hardware through side-channel signals such as power draw. We can exploit these signals to infer the state of ongoing computational tasks without having direct access to the device. This paper investigates the application of recent deep learning techniques to side-channel analysis in both classification of machine state and anomaly detection. We use real data collected from three different devices: an Arduino, a Raspberry Pi, and a Siemens PLC. For classification we compare the performance of a Multi-Layer Perceptron and a Long Short-Term Memory classifiers. Both achieve near-perfect accuracy on binary classification and around 90% accuracy on a multi-class problem. For anomaly detection we explore an autoencoder based model. Our experiments show the potential of using these deep learning techniques in side-channel analysis and cyber-attack detection.

Two-dimensional (2D) materials beyond graphene such as transition metal dichalcogenides (TMDs) have unique mechanical, optical and electronic properties with promising applications in flexible devices, catalysis and sensing. Optical imaging of TMDs using photoluminescence and Raman spectroscopy can reveal the effects of structure, strain, doping, edge states and surface functionalization from materials to bioscience. However, Raman signals are inherently weak and so far have been limited in spatial resolution in TMDs to a few hundred nanometres which is much larger than the intrinsic scale of these effects. Here we overcome the diffraction limit by using resonant tip-enhanced Raman scattering (TERS) of few-layer MoS2 and obtain nanoscale optical images with ~20 nm spatial resolution. This becomes possible due to electric field enhancement in an optimized subnanometre-gap resonant tip-substrate configuration. We investigate the limits of signal enhancement by varying the tip-sample gap with sub-Angstrom precision and observe a quantum quenching behavior, as well as a Schottky-Ohmic transition, for subnanometre gaps, which enable surface mapping based on this new contrast mechanism. This quantum regime of plasmonic gap-mode enhancement with a few nanometre thick MoS2 junction may be used for designing new quantum optoelectronic devices and sensors with a wide range of applications.