Search

Deriving interpretable prognostic features from deep-learning-based prognostic histopathology models remains a challenge. In this study, we developed a deep learning system (DLS) for predicting disease-specific survival for stage II and III colorectal cancer using 3652 cases (27,300 slides). When evaluated on two validation datasets containing 1239 cases (9340 slides) and 738 cases (7140 slides), respectively, the DLS achieved a 5-year disease-specific survival AUC of 0.70 (95% CI: 0.66–0.73) and 0.69 (95% CI: 0.64–0.72), and added significant predictive value to a set of nine clinicopathologic features. To interpret the DLS, we explored the ability of different human-interpretable features to explain the variance in DLS scores. We observed that clinicopathologic features such as T-category, N-category, and grade explained a small fraction of the variance in DLS scores (R2 = 18% in both validation sets). Next, we generated human-interpretable histologic features by clustering embeddings from a deep-learning-based image-similarity model and showed that they explained the majority of the variance (R2 of 73–80%). Furthermore, the clustering-derived feature most strongly associated with high DLS scores was also highly prognostic in isolation. With a distinct visual appearance (poorly differentiated tumor cell clusters adjacent to adipose tissue), this feature was identified by annotators with 87.0–95.5% accuracy. Our approach can be used to explain predictions from a prognostic deep learning model and uncover potentially-novel prognostic features that can be reliably identified by people for future validation studies. [ABSTRACT FROM AUTHOR]

Extraction of structured information from medical reports is a key enabling technology for generating large data sets of annotated digital slides needed for machine learning. Combined datasets of digital images and medical data alongside developments in AI/machine learning could make novel information accessible and quantifiable for human experts. Information could be hidden in images and is not findable by low level annotations. However, major progress might be expected with unsupervised learning using large numbers of digital slides associated with pathological, clinical and outcome data. To build the structured data source for pathological findings, we created a text mining system called SAAT (Semi Automated Annotation Tool). This uses an ontology-based term extraction and a decision tree built with a manually curated classification system. It extracts from pathology reports the ICD-10 and ICD-O Codes, TNM classification and immunohistochemistry results, e.g. hormone receptor status. A visual editor is used by experts to generate, modify and evaluate the rules. A reference dataset, excluded from the evaluation, is used to measure performance. 1.4 mio pathology reports generated over 23 years have been processed. We achieved F-Scores of 89.7% for ICD-10 and 94.78% for ICD-O classification. For extraction of the Tumor Stating and the receptors we achieved an F-Score range between 81.8 and 96.8%. SAAT is an open source tool and has been tested at 4 institutes in Austria, Germany and Switzerland. It can be used to generate a large scale data resource of digital slides linked with structured and cleaned histopathological information for machine learning. [ABSTRACT FROM AUTHOR]

In digital pathology the adoption of open source software has several positive implications, e.g. in the implementation of standards for data representation and the capacity of integration with other software and platforms. Furthermore the development of machine learning in digital pathology requires open access to large data sets of annotated training data sets. There are several open source tools available to process clinical and molecular data providing valuable annotations for whole slide images. However the utilization of open source software tools requires high technical skills for installation and configuration. To minimize this complexity we developed the software as a service (SaaS) framework BIBBOX (Basic Infrastructure Building Box). We build a box ready for digital pathology including whole slide management and viewers (e.g. Cytomine, OMERO, Girder), data analysis (QuPath, ASAP, SlideJ, HistomicsTK) and integrate these with other open source software for data management, analysis and sharing. "Digital pathology in a BOX" is available either as a virtual machine, or as a "real box" including a data management server, network interfaces and computing nodes. "Digital pathology in a BOX" can be used to organize federated whole slide image collections and/or as a standalone unit supporting the integration of digital pathology software with biobanking, clinical and life science tools. Beside this "Digital pathology in a BOX" may be used as informatics platform for education and training, and as a starter kit for digital pathology and biobanks in low and middle income countries. [ABSTRACT FROM AUTHOR]