Search

Supplemental Table S1 Patient characteristics for the National Lung Screening Trial (NLST) and The Cancer Genome Atlas (TCGA) datasets.

Supplemental Figure S8 Volcano plots of gene set enrichment analysis results correlating mRNA expression level with stroma nuclei density and karyorrhexis density, respectively.

The spatial organization of different types of cells in tumor tissues reveals important information about the tumor microenvironment (TME). To facilitate the study of cellular spatial organization and interactions, we developed Histology-based Digital-Staining, a deep learning-based computation model, to segment the nuclei of tumor, stroma, lymphocyte, macrophage, karyorrhexis, and red blood cells from standard hematoxylin and eosin–stained pathology images in lung adenocarcinoma. Using this tool, we identified and classified cell nuclei and extracted 48 cell spatial organization-related features that characterize the TME. Using these features, we developed a prognostic model from the National Lung Screening Trial dataset, and independently validated the model in The Cancer Genome Atlas lung adenocarcinoma dataset, in which the predicted high-risk group showed significantly worse survival than the low-risk group (P = 0.001), with a HR of 2.23 (1.37–3.65) after adjusting for clinical variables. Furthermore, the image-derived TME features significantly correlated with the gene expression of biological pathways. For example, transcriptional activation of both the T-cell receptor and programmed cell death protein 1 pathways positively correlated with the density of detected lymphocytes in tumor tissues, while expression of the extracellular matrix organization pathway positively correlated with the density of stromal cells. In summary, we demonstrate that the spatial organization of different cell types is predictive of patient survival and associated with the gene expression of biological pathways.Significance:These findings present a deep learning-based analysis tool to study the TME in pathology images and demonstrate that the cell spatial organization is predictive of patient survival and is associated with gene expression.See related commentary by Rodriguez-Antolin, p. 1912

Microscopic examination of pathology slides is essential to disease diagnosis and biomedical research; however, traditional manual examination of tissue slides is laborious and subjective. Tumor whole-slide image (WSI) scanning is becoming part of routine clinical procedure and produces massive data that capture tumor histological details at high resolution. Furthermore, the rapid development of deep learning algorithms has significantly increased the efficiency and accuracy of pathology image analysis. In light of this progress, digital pathology is fast becoming a powerful tool to assist pathologists.Studying tumor tissue and its surrounding microenvironment provides critical insight into tumor initiation, progression, metastasis, and potential therapeutic targets. Nuclei segmentation and classification are critical to pathology image analysis, especially in characterizing and quantifying the tumor microenvironment (TME). Computational algorithms have been developed for nuclei segmentation and TME quantification within image patches; however, existing algorithms are computationally intensive and time-consuming for WSI analysis.In this study, we present Histology-based Detection using Yolo (HD-Yolo), a new method that significantly accelerates nuclei segmentation and TME quantification. We demonstrate that HD-Yolo outperforms existing methods for WSI analysis in nuclei detection and classification accuracy, as well as computation time.

Tissues such as the liver lobule, kidney nephron, and intestinal gland exhibit intricate patterns of zonated gene expression corresponding to distinct cell types and functions. To quantitatively understand zonation, it would be important to measure cellular or genetic features as a function of position along a zonal axis. While it is possible to manually count, characterize, and locate features in relation to the zonal axis, it is very difficult to do this for more than a few hundred instances. We addressed this challenge by developing a deep-learning-based quantification method called the “Tissue Positioning System” (TPS), which can automatically analyze zonation in the liver lobule as a model system. By using algorithms that identified vessels, classified vessels, and segmented zones based on the relative position along the portal vein to central vein axis, TPS was able to spatially quantify gene expression in mice with zone specific reporters. TPS could discern expression differences between zonal reporter strains, ages, and disease states. TPS could also reveal the zonal distribution of cells previously thought to be randomly distributed. The design principles of TPS could be generalized to other tissues to explore the biology of zonation.The software is available athttps://github.com/yunguan-wang/Tissue_positioning_system.

Polyploidy, the duplication of the entire genome within a single cell, is a significant characteristic of cells in many tissues, including the liver. The quantification of hepatic ploidy typically relies on flow cytometry and immunofluorescence (IF) imaging, which are not widely available in clinical settings due to high financial and time costs. To improve accessibility for clinical samples, we developed a computational algorithm to quantify hepatic ploidy using hematoxylin-eosin (H&E) histopathology images, which are commonly obtained during routine clinical practice. Our algorithm uses a deep learning model to first segment and classify different types of cell nuclei in H&E images. It then determines cellular ploidy based on the relative distance between identified hepatocyte nuclei and determines nuclear ploidy using a fitted Gaussian mixture model. The algorithm can establish the total number of hepatocytes and their detailed ploidy information in a region of interest (ROI) on H&E images. This is the first successful attempt to automate ploidy analysis on H&E images. Our algorithm is expected to serve as an important tool for studying the role of polyploidy in human liver disease.

With the rapid development of image scanning techniques and visualization software, whole slide imaging (WSI) is becoming a routine diagnostic method. Accelerating clinical diagnosis from pathology images and automating image analysis efficiently and accurately remain significant challenges. Recently, deep learning algorithms have shown great promise in pathology image analysis, such as in tumor region identification, metastasis detection, and patient prognosis. Many machine learning algorithms, including convolutional neural networks, have been proposed to automatically segment pathology images. Among these algorithms, segmentation deep learning algorithms such as fully convolutional networks stand out for their accuracy, computational efficiency, and generalizability. Thus, deep learning–based pathology image segmentation has become an important tool in WSI analysis. In this review, the pathology image segmentation process using deep learning algorithms is described in detail. The goals are to provide quick guidance for implementing deep learning into pathology image analysis and to provide some potential ways of further improving segmentation performance. Although there have been previous reviews on using machine learning methods in digital pathology image analysis, this is the first in-depth review of the applications of deep learning algorithms for segmentation in WSI analysis.

Objective To provide an open-source software package for determining temporal correlations between disease states using longitudinal electronic medical records (EMR). Materials and Methods We have developed an R-based package, Disease Correlation Network (DCN), which builds retrospective matched cohorts from longitudinal medical records to assess for significant temporal correlations between diseases using two independent methodologies: Cox proportional hazards regression and random forest survival analysis. This optimizable package has the potential to control for relevant confounding factors such as age, gender, and other demographic and medical characteristics. Output is presented as a DCN which may be analyzed using a JavaScript-based interactive visualization tool for users to explore statistically significant correlations between disease states of interest using graph-theory-based network topology. Results We have applied this package to a longitudinal dataset at Loyola University Chicago Medical Center with 654 084 distinct initial diagnoses of 51 conditions in 175 539 patients. Over 90% of disease correlations identified are supported by literature review. DCN is available for download at https://github.com/qunfengdong/DCN. Conclusions DCN allows screening of EMR data to identify potential relationships between chronic disease states. This data may then be used to formulate novel research hypotheses for further characterization of these relationships.

Human microbiome consists of trillions of microorganisms. Microbiota can modulate the host physiology through molecule and metabolite interactions. Integrating microbiome and metabolomics data have the potential to predict different diseases more accurately. Yet, most datasets only measure microbiome data but without paired metabolome data. Here, we propose a novel integrative modeling framework, Microbiome-based Supervised Contrastive Learning Framework (MB-SupCon). MB-SupCon integrates microbiome and metabolome data to generate microbiome embeddings, which can be used to improve the prediction accuracy in datasets that only measure microbiome data. As a proof of concept, we applied MB-SupCon on 720 samples with paired 16S microbiome data and metabolomics data from patients with type 2 diabetes. MB-SupCon outperformed existing prediction methods and achieves high average prediction accuracies for insulin resistance status (84.62%), sex (78.98%), and race (80.04%). Moreover, the microbiome embeddings form separable clusters for different covariate groups in the lower-dimensional space, which enhances data visualization. We also applied MB-SupCon on a large inflammatory bowel disease study and observed similar advantages. Thus, MB-SupCon could be broadly applicable to improve microbiome prediction models in multi-omics disease studies.

Accelerometers provide continuous measurements of human movements. Utilizing their data exhibits great potential for monitoring or predicting human health outcomes. In this direction, researchers are active in developing machine-learning algorithms. However, compared to understanding deep learning models, visualizing and interpreting such models is equally important but has been less well-developed. To address these limitations, we constructed an online software tool to visualize the hidden states inferred from the deep learning models and to compare different cohorts based on outcome-associated temporal profiles. We demonstrated the utility of the model using the National Health and Nutrition Examination Survey (NHANES) dataset. Using this tool, we discovered that two time periods of daily activity contribute to significant differences in 5-year mortalities. To disseminate the software to the broader community for the analysis of accelerometer data, we provide the work as open-source code at https://github.com/jyan97/HiddenVis.

This study aims to develop an artificial intelligence (AI)-based model to assist radiologists in pneumoconiosis screening and staging using chest radiographs. The model, based on chest radiographs, was developed using a training cohort and validated using an independent test cohort. Every image in the training and test datasets were labeled by experienced radiologists in a double-blinded fashion. The computational model started by segmenting the lung field into six subregions. Then, convolutional neural network classification model was used to predict the opacity level for each subregion respectively. Finally, the diagnosis for each subject (normal, stage I, II, or III pneumoconiosis) was determined by summarizing the subregion-based prediction results. For the independent test cohort, pneumoconiosis screening accuracy was 0.973, with both sensitivity and specificity greater than 0.97. The accuracy for pneumoconiosis staging was 0.927, better than that achieved by two groups of radiologists (0.87 and 0.84, respectively). This study develops a deep learning-based model for screening and staging of pneumoconiosis using man-annotated chest radiographs. The model outperformed two groups of radiologists in the accuracy of pneumoconiosis staging. This pioneer work demonstrates the feasibility and efficiency of AI-assisted radiography screening and diagnosis in occupational lung diseases.

The spatial organization of different types of cells in tumor tissues reveals important information about the tumor microenvironment (TME). To facilitate the study of cellular spatial organization and interactions, we developed Histology-based Digital-Staining, a deep learning-based computation model, to segment the nuclei of tumor, stroma, lymphocyte, macrophage, karyorrhexis, and red blood cells from standard hematoxylin and eosin–stained pathology images in lung adenocarcinoma. Using this tool, we identified and classified cell nuclei and extracted 48 cell spatial organization-related features that characterize the TME. Using these features, we developed a prognostic model from the National Lung Screening Trial dataset, and independently validated the model in The Cancer Genome Atlas lung adenocarcinoma dataset, in which the predicted high-risk group showed significantly worse survival than the low-risk group (P = 0.001), with a HR of 2.23 (1.37–3.65) after adjusting for clinical variables. Furthermore, the image-derived TME features significantly correlated with the gene expression of biological pathways. For example, transcriptional activation of both the T-cell receptor and programmed cell death protein 1 pathways positively correlated with the density of detected lymphocytes in tumor tissues, while expression of the extracellular matrix organization pathway positively correlated with the density of stromal cells. In summary, we demonstrate that the spatial organization of different cell types is predictive of patient survival and associated with the gene expression of biological pathways. Significance: These findings present a deep learning-based analysis tool to study the TME in pathology images and demonstrate that the cell spatial organization is predictive of patient survival and is associated with gene expression. See related commentary by Rodriguez-Antolin, p. 1912

The spatial organization of different types of cells in tumor tissues reveals important information about the tumor microenvironment (TME). In order to facilitate the study of cellular spatial organization and interactions, we developed a comprehensive nuclei segmentation and classification tool to characterize the TME from standard Hematoxylin and Eosin (H&E)-stained pathology images. This tool can computationally "stain" different types of cell nuclei in H&E pathology images to facilitate pathologists in analyzing the TME. A Mask Regional-Convolutional Neural Network (Mask-RCNN) model was developed to segment the nuclei of tumor, stromal, lymphocyte, macrophage, karyorrhexis and red blood cells in lung adenocarcinoma (ADC). Using this tool, we identified and classified cell nuclei and extracted 48 cell spatial organization-related features that characterize the TME. Using these features, we developed a prognostic model from the National Lung Screening Trial dataset, and independently validated the model in The Cancer Genome Atlas (TCGA) lung ADC dataset, in which the predicted high-risk group showed significantly worse survival than the low-risk group (pv= 0.001), with a hazard ratio of 2.23 [1.37-3.65] after adjusting for clinical variables. Furthermore, the image-derived TME features were significantly correlated with the gene expression of biological pathways. For example, transcription activation of both the T-cell receptor (TCR) and Programmed cell death protein 1 (PD1) pathways was positively correlated with the density of detected lymphocytes in tumor tissues, while expression of the extracellular matrix organization pathway was positively correlated with the density of stromal cells. This study developed a deep learning-based analysis tool to dissect the TME from tumor tissue images. Using this tool, we demonstrated that the spatial organization of different cell types is predictive of patient survival and associated with the gene expression of biological pathways. Although developed from the pathology images of lung ADC, this model can be adapted into other types of cancers. Funding Statement: This work was partially supported by the National Institutes of Health [5R01CA152301, P50CA70907, 1R01GM115473, and 1R01CA172211], and the Cancer Prevention and Research Institute of Texas [RP190107 and RP180805]. Declaration of Interests: The authors declare that they have no competing interests. Ethics Approval Statement:Not required; date is available online.

Melanoma is the most deadly form of skin cancer worldwide. Many efforts have been made for early detection of melanoma with deep learning based on dermoscopic images. It is crucial to identify the specific lesion patterns for accurate diagnosis of melanoma. However, the common lesion patterns are not consistently present and cause sparse label problems in the data. In this paper, we propose a multi-task U-Net model to automatically detect lesion attributes of melanoma. The network includes two tasks, one is the classification task to classify if the lesion attributes present, and the other is the segmentation task to segment the attributes in the images. Our multi-task U-Net model achieves a Jaccard index of 0.433 on official test data of ISIC 2018 Challenges task 2, which ranks the 5th place on the final leaderboard.

Skin lesion is a severe disease globally. Early detection of melanoma in dermoscopy images significantly increases the survival rate. However, the accurate recognition of skin lesion is extremely challenging manually visualization. Hence, reliable automatic classification of skin lesions is meaningful to improve pathologists’ accuracy and efficiency. In this paper, we proposed a two-stage method to combine deep learning features and clinical criteria representations to address skin lesion automated diagnosis task. Stage 1 - feature encoding: Modified deep convolutional neural networks (CNNs, in this paper, we used Dense201 and Res50) were fine-tuned to extract rich image global features. To avoid hair noisy, we developed a lesion segmentation U-Net to mask out the decisive regions and used the masked image as CNNs inputs. In addition, color features, texture features and morphological features were exacted based on clinical criteria; Stage 2 - features fusion: LightGBM was used to select the salient features and model parameters, predicting diagnosis confidence for each category. The proposed deep learning frameworks were evaluated on the ISIC 2018 dataset. Experimental results show the promising accuracies of our frameworks were achieved.

Melanoma is the most deadly form of skin cancer worldwide. Many efforts have been made for early detection of melanoma. The International Skin Imaging Collaboration (ISIC) hosted the 2018 Challenges to help the diagnosis of melanoma based on dermoscopic images. In this paper, we describe our solutions for the task 2 of ISIC 2018 Challenges. We present two deep learning approaches to automatically detect lesion attributes of melanoma, one is a multi-task U-Net model and the other is a Mask R-CNN based model. Our multi-task U-Net model achieved a Jaccard index of 0.433 on official test data, which ranks the 5th place on the final leaderboard. The code for our solutions is publicly available.

Objective: Accurate diagnosis and prognosis are essential in lung cancer treatment selection and planning. With the rapid advance of medical imaging technology, whole slide imaging (WSI) in pathology is becoming a routine clinical procedure. An interplay of needs and challenges exists for computer-aided diagnosis based on accurate and efficient analysis of pathology images. Recently, artificial intelligence, especially deep learning, has shown great potential in pathology image analysis tasks such as tumor region identification, prognosis prediction, tumor microenvironment characterization, and metastasis detection. Materials and Methods: In this review, we aim to provide an overview of current and potential applications for AI methods in pathology image analysis, with an emphasis on lung cancer. Results: We outlined the current challenges and opportunities in lung cancer pathology image analysis, discussed the recent deep learning developments that could potentially impact digital pathology in lung cancer, and summarized the existing applications of deep learning algorithms in lung cancer diagnosis and prognosis. Discussion and Conclusion: With the advance of technology, digital pathology could have great potential impacts in lung cancer patient care. We point out some promising future directions for lung cancer pathology image analysis, including multi-task learning, transfer learning, and model interpretation.

Bacterial community composition in blood-sucking arthropods can shift dramatically across time and space. We used 16S rRNA gene amplification and pyrosequencing to investigate the relative impact of vertebrate host-related, arthropod-related and environmental factors on bacterial community composition in fleas and ticks collected from rodents in southern Indiana (USA). Bacterial community composition was largely affected by arthropod identity, but not by the rodent host or environmental conditions. Specifically, the arthropod group (fleas vs ticks) determined the community composition of bacteria, where bacterial communities of ticks were less diverse and more dependent on arthropod traits--especially tick species and life stage--than bacterial communities of fleas. Our data suggest that both arthropod life histories and the presence of arthropod-specific endosymbionts may mask the effects of the vertebrate host and its environment.

Rationale: The ability to determine the respiratory microbiome from bronchoalveolar lavage (BAL) is controversial due to oral contamination during bronchoscopy. To determine if processing BAL improved the ability to distinguish lung from oral wash microbiomes, we compared the oral and lung microbiome from whole and acellular BAL in 34 subjects. Methods: Oral wash samples and bronchoscopy with BAL were performed. After removing an aliquot of whole unprocessed BAL, the remaining was centrifuged at 1,500 rpm for 10 minutes and the supernatant harvested as acellular BAL. DNA was isolated from 2 ml of oral wash and 5 ml of acellular and whole BAL using the Qiagen DNeasy kit (Qiagen, Valencia, CA). DNA encoding 16s ribosomal RNA was amplified using NEB Phusion enzyme (NEB, Ipswich, MA) and degenerate of eubacterial primers that amplify the variable regions 1 through 3. DNA sequencing was performed using 454 sequencing. Results: The amount of genomic DNA isolated from whole BAL was 100-fold higher compared with acellular BAL. Despite this, the number of high-quality genera bacterial reads (RDP classifier > 90%) was identical between the two groups (whole BAL, 4,903 ± 3,209; acellular BAL, 4,245 ± 2,632). Bray-Curtis distances between oral wash and acellular BAL were significantly higher than the distance between whole BAL and oral wash (P = 0.000035), indicating acellular BAL was significantly more different from oral wash compared with whole BAL. Visually, the overlap between oral wash and whole BAL can be seen on the dendogram in Figure 1 (BA = acellular BAL; BW = whole BAL; OR = oral wash). Figure 1. Dendogram showing clustering using bray-curtis dissimilarity at the genus level. Blue = acellular bronchoalveolar lavage (BAL); red = whole BAL; green = oral wash. Conclusions: The difference between acellular BAL and oral wash is significantly greater than the difference between whole BAL and oral wash. We suggest this represents greater contamination of whole BAL by upper airway organisms. We speculate acellular BAL provides a better representation of the true lower respiratory tract microbiome.

Diabetes mellitus is a major risk factor for chronic periodontitis. We investigated the effects of type 2 diabetes on the subgingival plaque bacterial composition by applying culture-independent 16S rDNA sequencing to periodontal bacteria isolated from four groups of volunteers: non-diabetic subjects without periodontitis, non-diabetic subjects with periodontitis, type 2 diabetic patients without periodontitis, and type 2 diabetic patients with periodontitis. A total of 71,373 high-quality sequences were produced from the V1-V3 region of 16S rDNA genes by 454 pyrosequencing. Those 16S rDNA sequences were classified into 16 phyla, 27 classes, 48 orders, 85 families, 126 genera, and 1141 species-level OTUs. Comparing periodontally healthy samples with periodontitis samples identified 20 health-associated and 15 periodontitis-associated OTUs. In the subjects with healthy periodontium, the abundances of three genera (Prevotella, Pseudomonas, and Tannerella) and nine OTUs were significantly different between diabetic patients and their non-diabetic counterparts. In the subjects carrying periodontitis, the abundances of three phyla (Actinobacteria, Proteobacteria, and Bacteriodetes), two genera (Actinomyces and Aggregatibacter), and six OTUs were also significantly different between diabetics and non-diabetics. Our results show that type 2 diabetes mellitus could alter the bacterial composition in the subgingival plaque.

Clinical urine specimens are usually considered to be sterile when they do not yield uropathogens using standard clinical cultivation procedures. Our aim was to test if the adult female bladder might contain bacteria that are not identified by these routine procedures. An additional aim was to identify and recommend the appropriate urine collection method for the study of bacterial communities in the female bladder. Consenting participants who were free of known urinary tract infection provided urine samples by voided, transurethral, and/or suprapubic collection methods. The presence of bacteria in these samples was assessed by bacterial culture, light microscopy, and 16S rRNA gene sequencing. Bacteria that are not or cannot be routinely cultivated (hereinafter called uncultivated bacteria) were common in voided urine, urine collected by transurethral catheter (TUC), and urine collected by suprapubic aspirate (SPA), regardless of whether the subjects had urinary symptoms. Voided urine samples contained mixtures of urinary and genital tract bacteria. Communities identified in parallel urine samples collected by TUC and SPA were similar. Uncultivated bacteria are clearly present in the bladders of some women. It remains unclear if these bacteria are viable and/or if their presence is relevant to idiopathic urinary tract conditions.

Lactobacillus-dominated vaginal microbiotas are associated with reproductive health and STI resistance in women, whereas altered microbiotas are associated with bacterial vaginosis (BV), STI risk and poor reproductive outcomes. Putative vaginal taxa have been observed in male first-catch urine, urethral swab and coronal sulcus (CS) specimens but the significance of these observations is unclear. We used 16 S rRNA sequencing to characterize the microbiota of the CS and urine collected from 18 adolescent men over three consecutive months. CS microbiotas of most participants were more stable than their urine microbiotas and the composition of CS microbiotas were strongly influenced by circumcision. BV-associated taxa, including Atopobium, Megasphaera, Mobiluncus, Prevotella and Gemella, were detected in CS specimens from sexually experienced and inexperienced participants. In contrast, urine primarily contained taxa that were not abundant in CS specimens. Lactobacilllus and Streptococcus were major urine taxa but their abundance was inversely correlated. In contrast, Sneathia, Mycoplasma and Ureaplasma were only found in urine from sexually active participants. Thus, the CS and urine support stable and distinct bacterial communities. Finally, our results suggest that the penis and the urethra can be colonized by a variety of BV-associated taxa and that some of these colonizations result from partnered sexual activity.

Inflammation and infection of bovine mammary glands, commonly known as mastitis, imposes significant losses each year in the dairy industry worldwide. While several different bacterial species have been identified as causative agents of mastitis, many clinical mastitis cases remain culture negative, even after enrichment for bacterial growth. To understand the basis for this increasingly common phenomenon, the composition of bacterial communities from milk samples was analyzed using culture independent pyrosequencing of amplicons of 16S ribosomal RNA genes (16S rDNA). Comparisons were made of the microbial community composition of culture negative milk samples from mastitic quarters with that of non-mastitic quarters from the same animals. Genomic DNA from culture-negative clinical and healthy quarter sample pairs was isolated, and amplicon libraries were prepared using indexed primers specific to the V1–V2 region of bacterial 16S rRNA genes and sequenced using the Roche 454 GS FLX with titanium chemistry. Evaluation of the taxonomic composition of these samples revealed significant differences in the microbiota in milk from mastitic and healthy quarters. Statistical analysis identified seven bacterial genera that may be mainly responsible for the observed microbial community differences between mastitic and healthy quarters. Collectively, these results provide evidence that cases of culture negative mastitis can be associated with bacterial species that may be present below culture detection thresholds used here. The application of culture-independent bacterial community profiling represents a powerful approach to understand long-standing questions in animal health and disease.