Search

Supplementary Figure 1 from Large-Scale Profiling of Archival Lymph Nodes Reveals Pervasive Remodeling of the Follicular Lymphoma Methylome

Supplementary Table 3 from Large-Scale Profiling of Archival Lymph Nodes Reveals Pervasive Remodeling of the Follicular Lymphoma Methylome

Supplementary Figure Legend from Large-Scale Profiling of Archival Lymph Nodes Reveals Pervasive Remodeling of the Follicular Lymphoma Methylome

Supplementary Table 2 from Large-Scale Profiling of Archival Lymph Nodes Reveals Pervasive Remodeling of the Follicular Lymphoma Methylome

Supplementary Table 1 from Large-Scale Profiling of Archival Lymph Nodes Reveals Pervasive Remodeling of the Follicular Lymphoma Methylome

Supplementary Table 5 from Large-Scale Profiling of Archival Lymph Nodes Reveals Pervasive Remodeling of the Follicular Lymphoma Methylome

Supplementary Table 4 from Large-Scale Profiling of Archival Lymph Nodes Reveals Pervasive Remodeling of the Follicular Lymphoma Methylome

Emerging technologies allow broad profiling of the cancer genome for differential DNA methylation relative to benign cells. Herein, bisulfite-modified DNA from lymph nodes with either reactive hyperplasia or follicular lymphoma (FL) were analyzed using a commercial C/UpG genotyping assay. Two hundred fifty-nine differentially methylated targets (DMT) distributed among 183 unique genes were identified in FL. Comparison of matched formalin-fixed, paraffin-embedded and frozen surgical pathology replicates showed the complete preservation of the cancer methylome among differently archived tissue specimens. Analysis of the DMT profile is consistent with a pervasive epigenomic remodeling process in FL that affects predominantly nonlymphoid genes. [Cancer Res 2009;69(3):758–64]

Supplementary Table 6 from Large-Scale Profiling of Archival Lymph Nodes Reveals Pervasive Remodeling of the Follicular Lymphoma Methylome

Understanding the complex biology of the tumor microenvironment (TME) is necessary to understand the mechanisms of action of immuno-oncology therapies and to match the right therapies to the right patients. Multiplex immunofluorescence (mIF) is a useful technology that has tremendous potential to further our understanding of cancer patho-biology; however, tools that fully leverage the high dimensionality of this data are still in their infancy. We describe here a novel deep learning pipeline aimed to allow Graph-based Inspection of Tissues via Embeddings, GraphITE. GraphITE transforms mIF data into a graph representation, where unsupervised learning algorithms can be utilised to generate embeddings representing cellular ‘neighbourhoods’. The embeddings can be downprojected and explored for clustering analysis, and patterns can be mapped back to the image as well as interrogated for phenotypical, morphological, or structural distinctiveness. GraphITE supports the extraction of information not only on the phenotypes of individual cells or the relationships between specific cell types, but is able to characterize cell neighborhoods to look for more complex interactions, thereby allowing pathologists and data scientists to explore mIF data sets, uncovering patterns that are otherwise obscured by the high-dimensionality of the data. In this work, we showcase the current setup of the system, going from raw input data all the way to a user friendly exploration tool. Using this tool, we show how the data can be navigated in a way previously not possible.

Supplemental Digital Content is available in the text., Advances in the quality of whole-slide images have set the stage for the clinical use of digital images in anatomic pathology. Along with advances in computer image analysis, this raises the possibility for computer-assisted diagnostics in pathology to improve histopathologic interpretation and clinical care. To evaluate the potential impact of digital assistance on interpretation of digitized slides, we conducted a multireader multicase study utilizing our deep learning algorithm for the detection of breast cancer metastasis in lymph nodes. Six pathologists reviewed 70 digitized slides from lymph node sections in 2 reader modes, unassisted and assisted, with a wash-out period between sessions. In the assisted mode, the deep learning algorithm was used to identify and outline regions with high likelihood of containing tumor. Algorithm-assisted pathologists demonstrated higher accuracy than either the algorithm or the pathologist alone. In particular, algorithm assistance significantly increased the sensitivity of detection for micrometastases (91% vs. 83%, P=0.02). In addition, average review time per image was significantly shorter with assistance than without assistance for both micrometastases (61 vs. 116 s, P=0.002) and negative images (111 vs. 137 s, P=0.018). Lastly, pathologists were asked to provide a numeric score regarding the difficulty of each image classification. On the basis of this score, pathologists considered the image review of micrometastases to be significantly easier when interpreted with assistance (P=0.0005). Utilizing a proof of concept assistant tool, this study demonstrates the potential of a deep learning algorithm to improve pathologist accuracy and efficiency in a digital pathology workflow.

Across all stages of drug discovery and development, experimental assays are performed to understand the effect of a drug or drug candidate—at the molecular, cellular, organ, or organism level. Imaging is a key technology in this process, and an imaging assay consisting of sample preparation, image acquisition, and image analysis provides a quantitative readout of a system. Historically, chemical assays have been the workhorse of early discovery, screening millions of compounds for a simple endpoint, whereas imaging was primarily used in lower throughput mechanistic studies. However, with development of high-throughput high-content microscopy platforms, the throughput of imaging assays now rivals chemical screens. Similarly, innovations in image analysis mean that robust quantitative conclusions can be derived from complex and multimodal image data, driving informed decision making later in the drug development process. For these reasons, imaging is widely used throughout the pharmaceutical industry and throughout multiple stages of the drug discovery process.

April 12, 2017 marked a significant day in the evolution of digital pathology in the United States, when the US Food and Drug Administration announced its approval of the Philips IntelliSite Pathology Solution for primary diagnosis in surgical pathology. Although this event is expected to facilitate more widespread adoption of whole slide imaging for clinical applications in the United States, it also raises a number of questions as to the means by which pathologists might choose to incorporate this technology into their clinical practice. This article from the College of American Pathologists Digital Pathology Committee reviews frequently asked questions on this topic and provides answers based on currently available information.

Introduction Machine learning (ML) models offer the potential to provide rich, quantitative characterizations of the tumor and tumor micro-environment (TME). Here we deployed a machine learning-based approach to the analysis of H&E images from HUDSON (NCT03334617), an AstraZeneca Phase II Platform clinical trial, to identify and quantify cellular composition and tissue architecture features in the TME that are associated with genomic alterations and time to progression on anti-PD(L)1 therapies. Methods PathAI previously trained ML models on non-small cell lung carcinoma (NSCLC) samples from commercial and clinical datasets to identify cell types and tissue regions within the TME. With no additional training, the models were deployed on 169 digitized whole slide images (WSIs) of H&E-stained biopsies from an international, multi-site AstraZeneca-sponsored Phase II clinical trial of novel anti-cancer agents in subjects with metastatic NSCLC. Biopsies were across multiple body sites, and taken both pre- and post-checkpoint progression. ML models generated human interpretable features (HIFs) that characterize the cell composition and tissue architecture from each biopsied sample. HIFs from baseline samples that met minimum image quality thresholds (n=89) were clustered to reduce redundancy and were tested for association with weeks to progression on anti-PD(L)1 therapy using Cox regression analysis. Results The PathAI ML models were successfully deployed on WSIs from the HUDSON clinical trial. Following correction for biopsy timing and location, a total of 59 HIFs were found to be significantly associated (p Conclusions PathAI models were able to identify TME-associated features from WSIs from a Phase II clinical trial which were associated with therapy failure and genomic alterations. These results suggest the power of deploying pre-trained ML-based systems in a clinical trial setting to identify pathobiological features associated with tumor characteristics and time to progression from only H&E images. Citation Format: Laura Dillon, Marylens Hernandez, Ben Glass, Guillaume Chhor, Sara Hoffman, Varsha Chinnaobireddy, Sai Chowdary Gullapally, Kris Sachsenmeier, Andy Beck, Jason Hipp. Deep learning identifies pathobiological features within H&E images associated with genomic alterations and progression on anti-PD(L)1 in HUDSON, an AstraZeneca-sponsored Phase II clinical trial [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr LB016.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

The increasing availability of large institutional and public histopathology image datasets is enabling the searching of these datasets for diagnosis, research, and education. Though these datasets typically have associated metadata such as diagnosis or clinical notes, even carefully curated datasets rarely contain annotations of the location of regions of interest on each image. Because pathology images are extremely large (up to 100,000 pixels in each dimension), further laborious visual search of each image may be needed to find the feature of interest. In this paper, we introduce a deep learning based reverse image search tool for histopathology images: Similar Medical Images Like Yours (SMILY). We assessed SMILY's ability to retrieve search results in two ways: using pathologist-provided annotations, and via prospective studies where pathologists evaluated the quality of SMILY search results. As a negative control in the second evaluation, pathologists were blinded to whether search results were retrieved by SMILY or randomly. In both types of assessments, SMILY was able to retrieve search results with similar histologic features, organ site, and prostate cancer Gleason grade compared with the original query. SMILY may be a useful general-purpose tool in the pathologist's arsenal, to improve the efficiency of searching large archives of histopathology images, without the need to develop and implement specific tools for each application., 23 Pages with 6 figures and 3 tables. The file also has 6 pages of supplemental material. Improved figure resolution, edited metadata

Introduction Machine learning models offer the potential to provide rich, quantitative characterizations of the tumor and tumor micro-environment (TME); however, historically it has been difficult to generalize trained models to new sets of clinical trial samples from trials not used in training. Here we evaluate the ability to deploy a machine learning based model (ML Model) for the identification of non-small cell lung tissue regions and lymphocytes within the tumor and TME on H&E stained images from clinical trial samples with no additional model training. Methods The ML model was previously trained on both squamous cell carcinoma and lung adenocarcinoma non-small cell lung carcinoma (NSCLC) samples from commercial and clinical datasets. The ML model was deployed on an AstraZeneca-sponsored phase II clinical trial of novel anti-cancer agents in patients with metastatic NSCLC. In order to validate the predictions of lymphocytes from the H&E stained images, we established a reference dataset for manual vs digital concordance consisting of 300, 150 × 150-micron–sized “frames” sampled from the trial dataset, removing frames of inadequate tissue quality or with presence of artifacts. For each frame, we collected exhaustive annotations from 5 pathologists to produce quantitative estimates of lymphocytes. Altogether, 43,932 annotations were collected and used to compute pathologist consensus scores for each frame. These scores were then correlated with each individual pathologist (inter-reader agreement) and with the PathAI-derived automated scores for evaluation of manual vs digital agreement. Results The PathAI system was successfully deployed on 169 H&E stained images from the phase II clinical trial to exhaustively identify all tumor associated lymphocytes from each whole slide image. In total, PathAI classified 2,859,796 lymphocytes, with an average number of 16,922 lymphocytes per image. We used frames-based validation to determine the correlation between the automated scoring and consensus scoring from pathologists hand labeling individual lymphocytes within image frames. The PathAI platform showed strong correlation between reference-based consensus scores (r2 = 0.84, CI [0.80 – 0.87]) and the ML model, which was similar to the level of agreement achieved between individual pathologists (r2 = 0.80, CI [0.76 – 0.85]). Conclusions The PathAI system showed strong generalizability for the identification of lymphocytes within the tumor and TME from H&E stained images from NSCLC clinical trial samples. These results suggest the power of deploying ML-based systems broadly for the automated, single cell resolution characterization of disease pathology from clinical trial material. Citation Format: Ben Glass, Laura Dillon, Guillaume Chhor, Sara Hoffman, Varsha Chinnaobireddy, Sai Chowdary Gullapally, Andy Beck, Jason Hipp. Robust deployment of ML models quantifying the H&E tumor microenvironment in NSCLC subjects from an AstraZeneca-sponsored phase II clinical trial [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-072.

Introduction: In digital pathology, large amounts of time and money are spent annotating whole slide images (WSIs). By using an AI algorithm trained on thousands of WSIs, it is possible to remove the process of annotating from the workflow whilst simultaneously finding potential imaging features that correlate with drug response, molecular phenotypes, histologic features (e.g. Tertiary Lymphoid Structures), inflammation at the tumor-stroma interface, artifacts, etc. Method: A pipeline was constructed for ingesting WSI from the set of diagnostic slides available in the cancer genome archive (TCGA). The WSIs were masked for tissue areas and divided into patches extracted at multiple magnification levels. These patches were used to train a U-Net (initialised with a pre-trained ResNet34) to complete a self-supervised in-painting task and evaluated with perceptual loss using VGG16. After training, embeddings were extracted from the bottleneck layer of the U-Net, and these were evaluated for their capacity to cluster according to tissue type and magnification level. Results: The trained embeddings showed strong clustering by tissue type, magnification level, and separation of artefacts. The results were qualitatively evaluated by pathologist with consensus and exceeded the performance of a baseline pre-trained ImageNet model. Conclusion: These results highlight a novel methodology for AI algorithm development that removes the need for numerous pathologists to annotate pathology images by hand and converts their role to reviewing and quality-checking images. AI algorithms based on this approach can have a significant impact in the pathologic evaluation of tissue for oncology clinical trials by extracting a richer set of features than glass microscopy alone would permit. Furthermore, self-supervised methods, such as in-painting allow for the training of embeddings that encapsulate a richer understanding of the data and are consequently more repurposable than supervised/labelled approaches. Citation Format: Jason Hipp, Mona Xu, Lucas Bordeaux, Feng Gu, Carlos Pedrinaci, Khan Baykaner. Unsupervised learning of image embeddings enables new opportunities to extract novel information from digital pathology H&E images [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-076.

Characterization of the location and phenotype of cells in the tumor microenvironment (TME) is important to inform the development and monitoring of anti-cancer therapeutic interventions, especially immunotherapies designed to stimulate the immune system to have an anti-cancer effect. Multiplex immunofluorescence (mIF) imaging is being increasingly employed to simultaneously label multiple cell types and subtypes in the tumor microenvironment, but interpretation of these images to gain a robust understanding of tumor and immune cell interactions remains a complicated and challenging process. The rich phenotypic information contained in mIF images has to be taken into account with the spatial topology of the cells in order to be able to distil potential predictive indicators of patient response to therapies as well as prognostic indicators of outcome. While contemporary computational methods allow pathologists to view aggregated phenotypical information and cell interactions on a limited, generally one-to-one basis, these methods have been largely descriptive and geared toward addressing hypotheses as opposed to holistically leveraging the spatial and phenotypic data into a single predictive model. Additional methods are needed to provide a fuller picture of the spatial structure of the TME as captured in mIF images. In this work, we propose a novel pipeline that uses graphs generated from image analysis results and user-defined distance criteria to represent the tumor cellular microstructure. This graph-based approach complements existing mIF analysis techniques by providing information on the spatial, phenotypic, and morphological features of cells in the context of their neighborhood. These graphs subsequently enable characterization of protein expression in detail, description of interactions between individual cells or cell types and their neighbors, interactive tissue querying, and exploration of the cell-level biodiversity. The graph approach not only allows pathologists to efficiently interrogate data contained in mIF images in a hypothesis-driven manner, but importantly also supports more holistic data-driven approaches which, by leveraging state of the art graph convolutional neural networks to obtain numerical embeddings representing each graph and its nodes, enable additional downstream activities such as cell similarity search, and the development of predictive models for patient outcomes and response to therapies. Citation Format: Jason Hipp, Christopher Innocenti, Zhenning Zhang, Jake Cohen-Setton, Balaji Selvaraj, Michalis Frangos, Carlos Pedrinaci, Michael Surace, Laura Dillon, Khan Baykaner. Leveraging graphs to do novel hypothesis and data-driven research using multiplex immunofluorescence images [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PR-05.

Rendering cancer diagnoses from tissues is a highly complex process that requires many years of expert training. It involves challenging tasks for pathologists, such as identifying rare events in very large images (e.g., micrometastases in lymph nodes) or classifying subtle differences between normal vs tumor or among similar-looking tumors that have very different treatment plans. These tasks are typically more challenging for human cognition, and, consequently, over- and underdiagnoses are not uncommon, resulting in nonoptimal treatment selection. Deep learning, a type of machine intelligence, has been successfully applied to solve similar identification and classification problems for Gmail, Google Maps, Google Photos, Google Translate, etc., and may have the potential to solve these similar challenges in the practice of pathology and help improve patient's diagnoses. This talk will begin by providing a brief background on deep learning and how it has been utilized at Google for health care and non-health care applications. The latter half of the talk will discuss the potential of deep learning to improve the accuracy and efficiency of the pathology workflow while highlighting recent advances towards that goal. Citation Format: Jason D. Hipp, Martin Stumpe. Advancing cancer diagnostics with artificial intelligence [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2018; 2018 Apr 14-18; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2018;78(13 Suppl):Abstract nr PL02-01.