Search

The precision-based revolution in medicine continues to demand stratification of patients into smaller and more personalized subgroups. While genomic technologies have largely led this movement, diagnostic results can take days to weeks to generate. Management at, or closer to, the point of care still heavily relies on the subjective qualitative interpretation of clinical and diagnostic imaging findings. New and emerging technological advances in artificial intelligence (AI) now appear poised to help bring objectivity and precision to these traditionally qualitative analytic tools. In particular, one specific form of AI, known as deep learning, is achieving expert-level disease classifications in many areas of diagnostic medicine dependent on visual and image-based findings. Here, we briefly review concepts of deep learning, and more specifically recent developments in convolutional neural networks (CNNs), to highlight their transformative potential in personalized medicine and, in particular, diagnostic histopathology. Understanding the opportunities and challenges of these quantitative machine-based decision support tools is critical to their widespread introduction into routine diagnostics.

Background There is growing interest in utilizing artificial intelligence, and particularly deep learning, for computer vision in histopathology. While accumulating studies highlight expert-level performance of convolutional neural networks (CNNs) on focused classification tasks, most studies rely on probability distribution scores with empirically defined cutoff values based on post-hoc analysis. More generalizable tools that allow humans to visualize histology-based deep learning inferences and decision making are scarce. Results Here, we leverage t-distributed Stochastic Neighbor Embedding (t-SNE) to reduce dimensionality and depict how CNNs organize histomorphologic information. Unique to our workflow, we develop a quantitative and transparent approach to visualizing classification decisions prior to softmax compression. By discretizing the relationships between classes on the t-SNE plot, we show we can super-impose randomly sampled regions of test images and use their distribution to render statistically-driven classifications. Therefore, in addition to providing intuitive outputs for human review, this visual approach can carry out automated and objective multi-class classifications similar to more traditional and less-transparent categorical probability distribution scores. Importantly, this novel classification approach is driven by a priori statistically defined cutoffs. It therefore serves as a generalizable classification and anomaly detection tool less reliant on post-hoc tuning. Conclusion Routine incorporation of this convenient approach for quantitative visualization and error reduction in histopathology aims to accelerate early adoption of CNNs into generalized real-world applications where unanticipated and previously untrained classes are often encountered. Electronic supplementary material The online version of this article (10.1186/s12859-018-2184-4) contains supplementary material, which is available to authorized users.

Figures S1-S4. Additional data figures cited throughout the manuscript. (PDF 2768 kb)

Table S1. Distribution of tissue types and images used for the development of the CNN training dataset. (PDF 153 kb)

Brain tumors represent a diverse group of neoplasms with highly variable therapies and clinical outcomes. Early personalized clinical management and initiation of precision-based molecular studies still heavily relies on morphologic interpretation of hematoxylin and eosin (H&E)-stained slides. Unfortunately, due to its qualitative nature, histopathological classification is prone to well-recognized inter-observer variability. To overcome this limitation, we developed an objective morphology-based brain tumor classifier using a deep convolutional neural network (CNN). Our CNN is trained on a dataset of over one million pathologist- and molecularly-annotated image patches from H&E slides spanning over 20 common brain tumor classes. Importantly, our tool is fully automated, compatible with standard pathology workflows and provides prompt whole-slide annotation and lesion classification in under 5 minutes. The performance of our CNN-based tumor classifier is highly concordant with board-certified pathologists and confirmatory immunohistochemical stains. Testing reveals an area under the receiver operator characteristic (AUC) of >0.95 for multiple classification tasks, including lesion localizing and differentiating among different brain tumor classes. In certain scenarios, it also offers objective predictions of actionable molecular alterations (IDH mutations and 1p19q co-deletions). Lastly, we use cloud-computing to provide our classifier as a web-based tool capable of rendering timely second opinions and quality assurance to remote cancer centers requiring additional subspecialized neuropathological expertise. This study demonstrates the efficacy of utilizing artificial intelligence to create an autonomous histologic brain tumor classifier. Acutely, our compact tool aims to provide prompt, intra-operative information to help tailor surgical resections and personalized therapies. In the sub-acute setting, our CNN can provide objective triaging of molecular tests to help reduce diagnostic work-up times, costs and subjective interpretative errors. Our classifier thus has immediate translational potential as a rapid, precise and cost-effective tool to help guide personalized care in neuro-oncology.