Search

We introduce a pioneering approach that integrates pathology imaging with transcriptomics and proteomics to identify predictive histology features associated with critical clinical outcomes in cancer. We utilize 2,755 H&E-stained histopathological slides from 657 patients across 6 cancer types from CPTAC. Our models effectively recapitulate distinctions readily made by human pathologists: tumor vs. normal (AUROC = 0.995) and tissue-of-origin (AUROC = 0.979). We further investigate predictive power on tasks not normally performed from H&E alone, including TP53 prediction and pathologic stage. Importantly, we describe predictive morphologies not previously utilized in a clinical setting. The incorporation of transcriptomics and proteomics identifies pathway-level signatures and cellular processes driving predictive histology features. Model generalizability and interpretability is confirmed using TCGA. We propose a classification system for these tasks, and suggest potential clinical applications for this integrated human and machine learning approach. A publicly available web-based platform implements these models., Competing Interests: Declaration of interests The authors declare no competing interests., (Copyright © 2023 The Author(s). Published by Elsevier Inc. All rights reserved.)

Early diagnosis of Alzheimer's disease plays a pivotal role in patient care and clinical trials. In this study, we have developed a new approach based on 3D deep convolutional neural networks to accurately differentiate mild Alzheimer's disease dementia from mild cognitive impairment and cognitively normal individuals using structural MRIs. For comparison, we have built a reference model based on the volumes and thickness of previously reported brain regions that are known to be implicated in disease progression. We validate both models on an internal held-out cohort from The Alzheimer's Disease Neuroimaging Initiative (ADNI) and on an external independent cohort from The National Alzheimer's Coordinating Center (NACC). The deep-learning model is accurate, achieved an area-under-the-curve (AUC) of 85.12 when distinguishing between cognitive normal subjects and subjects with either MCI or mild Alzheimer's dementia. In the more challenging task of detecting MCI, it achieves an AUC of 62.45. It is also significantly faster than the volume/thickness model in which the volumes and thickness need to be extracted beforehand. The model can also be used to forecast progression: subjects with mild cognitive impairment misclassified as having mild Alzheimer's disease dementia by the model were faster to progress to dementia over time. An analysis of the features learned by the proposed model shows that it relies on a wide range of regions associated with Alzheimer's disease. These findings suggest that deep neural networks can automatically learn to identify imaging biomarkers that are predictive of Alzheimer's disease, and leverage them to achieve accurate early detection of the disease., (© 2022. The Author(s).)

Image-based analysis as a method for mutation detection can be advantageous in settings when tumor tissue is limited or unavailable for direct testing. In this study, we utilize two distinct and complementary machine-learning methods of analyzing whole-slide images for predicting mutated BRAF. In the first method, whole-slide images of melanomas from 256 patients were used to train a deep convolutional neural network to develop a fully automated model that first selects for tumor-rich areas (area under the curve = 0.96) and then predicts for mutated BRAF (area under the curve = 0.71). Saliency mapping was performed and revealed that pixels corresponding to nuclei were the most relevant to network learning. In the second method, whole-slide images were analyzed using a pathomics pipeline that first annotates nuclei and then quantifies nuclear features, showing that mutated BRAF nuclei were significantly larger and rounder than BRAF‒wild-type nuclei. Finally, we developed a model that combines clinical information, deep learning, and pathomics that improves the predictive performance for mutated BRAF to an area under the curve of 0.89. Not only does this provide additional insights on how BRAF mutations affect tumor structural characteristics, but machine learning‒based analysis of whole-slide images also has the potential to be integrated into higher-order models for understanding tumor biology., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)

Biopsies of inflammatory tissue contain a complex network of interacting cells, orchestrating the immune or autoimmune response. While standard histological examination can identify relationships, it is clear that a great amount of data on each slide is not quantitated or categorized in standard microscopic examinations. To deal with the huge amount of data present in biopsy tissue in an unbiased and comprehensive way, we have developed a deep learning algorithm to identify immune cells in biopsies of inflammatory lesions. We focused on T follicular helper (Tfh) cell subsets and B cells in dermatomyositis biopsy images. We achieved strong performance on detection and classification of cells, including the rare Tfh cell subsets present in the tissue. This algorithm could be used to perform distance mapping between cell types in tissue, and could be easily adapted to other disease states., (Copyright © 2022. Published by Elsevier B.V.)