Search

Artificial intelligence (AI) has been commoditized. It has evolved from a specialty resource to a readily accessible tool for cancer researchers. AI-based tools can boost research productivity in daily workflows, but can also extract hidden information from existing data, thereby enabling new scientific discoveries. Building a basic literacy in these tools is useful for every cancer researcher. Researchers with a traditional biological science focus can use AI-based tools through off-the-shelf software, whereas those who are more computationally inclined can develop their own AI-based software pipelines. In this article, we provide a practical guide for non-computational cancer researchers to understand how AI-based tools can benefit them. We convey general principles of AI for applications in image analysis, natural language processing and drug discovery. In addition, we give examples of how non-computational researchers can get started on the journey to productively use AI in their own work., (© 2024. Springer Nature Limited.)

In the spectrum of colorectal tumors, microsatellite-stable (MSS) tumors with DNA polymerase ε (POLE) mutations exhibit a hypermutated profile, holding the potential to respond to immunotherapy similarly to their microsatellite-instable (MSI) counterparts. Yet, due to their rarity and the associated testing costs, systematic screening for these mutations is not commonly pursued. Notably, the histopathological phenotype resulting from POLE mutations is theorized to resemble that of MSI. This resemblance not only could facilitate their detection by a transformer-based Deep Learning (DL) system trained on MSI pathology slides, but also indicates the possibility for MSS patients with POLE mutations to access enhanced treatment options, which might otherwise be overlooked. To harness this potential, we trained a Deep Learning classifier on a large dataset with the ground truth for microsatellite status and subsequently validated its capabilities for MSI and POLE detection across three external cohorts. Our model accurately identified MSI status in both the internal and external resection cohorts using pathology images alone. Notably, with a classification threshold of 0.5, over 75% of POLE driver mutant patients in the external resection cohorts were flagged as "positive" by a DL system trained on MSI status. In a clinical setting, deploying this DL model as a preliminary screening tool could facilitate the efficient identification of clinically relevant MSI and POLE mutations in colorectal tumors, in one go., (© 2024. The Author(s).)

Background: The field of Artificial Intelligence (AI) holds transformative potential in medicine. However, the lack of universal reporting guidelines poses challenges in ensuring the validity and reproducibility of published research studies in this field., Methods: Based on a systematic review of academic publications and reporting standards demanded by both international consortia and regulatory stakeholders as well as leading journals in the fields of medicine and medical informatics, 26 reporting guidelines published between 2009 and 2023 were included in this analysis. Guidelines were stratified by breadth (general or specific to medical fields), underlying consensus quality, and target research phase (preclinical, translational, clinical) and subsequently analyzed regarding the overlap and variations in guideline items., Results: AI reporting guidelines for medical research vary with respect to the quality of the underlying consensus process, breadth, and target research phase. Some guideline items such as reporting of study design and model performance recur across guidelines, whereas other items are specific to particular fields and research stages., Conclusions: Our analysis highlights the importance of reporting guidelines in clinical AI research and underscores the need for common standards that address the identified variations and gaps in current guidelines. Overall, this comprehensive overview could help researchers and public stakeholders reinforce quality standards for increased reliability, reproducibility, clinical validity, and public trust in AI research in healthcare. This could facilitate the safe, effective, and ethical translation of AI methods into clinical applications that will ultimately improve patient outcomes., (© 2024. The Author(s).)

Histopathology and genomic profiling are cornerstones of precision oncology and are routinely obtained for patients with cancer. Traditionally, histopathology slides are manually reviewed by highly trained pathologists. Genomic data, on the other hand, is evaluated by engineered computational pipelines. In both applications, the advent of modern artificial intelligence methods, specifically machine learning (ML) and deep learning (DL), have opened up a fundamentally new way of extracting actionable insights from raw data, which could augment and potentially replace some aspects of traditional evaluation workflows. In this review, we summarize current and emerging applications of DL in histopathology and genomics, including basic diagnostic as well as advanced prognostic tasks. Based on a growing body of evidence, we suggest that DL could be the groundwork for a new kind of workflow in oncology and cancer research. However, we also point out that DL models can have biases and other flaws that users in healthcare and research need to know about, and we propose ways to address them., (© 2024. The Author(s).)

The technological progress in artificial intelligence (AI) has massively accelerated since 2022, with far-reaching implications for oncology and cancer research. Large language models (LLMs) now perform at human-level competency in text processing. Notably, both text and image processing networks are increasingly based on transformer neural networks. This convergence enables the development of multimodal AI models that take diverse types of data as an input simultaneously, marking a qualitative shift from specialized niche models which were prevalent in the 2010s. This editorial summarizes these developments, which are expected to impact precision oncology in the coming years., (© 2024. The Author(s).)

Background: The objective of this comprehensive pan-cancer study is to evaluate the potential of deep learning (DL) for molecular profiling of multi-omic biomarkers directly from hematoxylin and eosin (H&E)-stained whole slide images., Methods: A total of 12,093 DL models predicting 4031 multi-omic biomarkers across 32 cancer types were trained and validated. The study included a broad range of genetic, transcriptomic, and proteomic biomarkers, as well as established prognostic markers, molecular subtypes, and clinical outcomes., Results: Here we show that 50% of the models achieve an area under the curve (AUC) of 0.644 or higher. The observed AUC for 25% of the models is at least 0.719 and exceeds 0.834 for the top 5%. Molecular profiling with image-based histomorphological features is generally considered feasible for most of the investigated biomarkers and across different cancer types. The performance appears to be independent of tumor purity, sample size, and class ratio (prevalence), suggesting a degree of inherent predictability in histomorphology., Conclusions: The results demonstrate that DL holds promise to predict a wide range of biomarkers across the omics spectrum using only H&E-stained histological slides of solid tumors. This paves the way for accelerating diagnosis and developing more precise treatments for cancer patients., (© 2024. The Author(s).)

A knowledge gap persists between machine learning (ML) developers (e.g., data scientists) and practitioners (e.g., clinicians), hampering the full utilization of ML for clinical data analysis. We investigated the potential of the ChatGPT Advanced Data Analysis (ADA), an extension of GPT-4, to bridge this gap and perform ML analyses efficiently. Real-world clinical datasets and study details from large trials across various medical specialties were presented to ChatGPT ADA without specific guidance. ChatGPT ADA autonomously developed state-of-the-art ML models based on the original study's training data to predict clinical outcomes such as cancer development, cancer progression, disease complications, or biomarkers such as pathogenic gene sequences. Following the re-implementation and optimization of the published models, the head-to-head comparison of the ChatGPT ADA-crafted ML models and their respective manually crafted counterparts revealed no significant differences in traditional performance metrics (p ≥ 0.072). Strikingly, the ChatGPT ADA-crafted ML models often outperformed their counterparts. In conclusion, ChatGPT ADA offers a promising avenue to democratize ML in medicine by simplifying complex data analyses, yet should enhance, not replace, specialized training and resources, to promote broader applications in medical research and practice., (© 2024. The Author(s).)

Deep Learning (DL) can predict biomarkers from cancer histopathology. Several clinically approved applications use this technology. Most approaches, however, predict categorical labels, whereas biomarkers are often continuous measurements. We hypothesize that regression-based DL outperforms classification-based DL. Therefore, we develop and evaluate a self-supervised attention-based weakly supervised regression method that predicts continuous biomarkers directly from 11,671 images of patients across nine cancer types. We test our method for multiple clinically and biologically relevant biomarkers: homologous recombination deficiency score, a clinically used pan-cancer biomarker, as well as markers of key biological processes in the tumor microenvironment. Using regression significantly enhances the accuracy of biomarker prediction, while also improving the predictions' correspondence to regions of known clinical relevance over classification. In a large cohort of colorectal cancer patients, regression-based prediction scores provide a higher prognostic value than classification-based scores. Our open-source regression approach offers a promising alternative for continuous biomarker analysis in computational pathology., (© 2024. The Author(s).)

Background: Pretraining labeled datasets, like ImageNet, have become a technical standard in advanced medical image analysis. However, the emergence of self-supervised learning (SSL), which leverages unlabeled data to learn robust features, presents an opportunity to bypass the intensive labeling process. In this study, we explored if SSL for pretraining on non-medical images can be applied to chest radiographs and how it compares to supervised pretraining on non-medical images and on medical images., Methods: We utilized a vision transformer and initialized its weights based on the following: (i) SSL pretraining on non-medical images (DINOv2), (ii) supervised learning (SL) pretraining on non-medical images (ImageNet dataset), and (iii) SL pretraining on chest radiographs from the MIMIC-CXR database, the largest labeled public dataset of chest radiographs to date. We tested our approach on over 800,000 chest radiographs from 6 large global datasets, diagnosing more than 20 different imaging findings. Performance was quantified using the area under the receiver operating characteristic curve and evaluated for statistical significance using bootstrapping., Results: SSL pretraining on non-medical images not only outperformed ImageNet-based pretraining (p < 0.001 for all datasets) but, in certain cases, also exceeded SL on the MIMIC-CXR dataset. Our findings suggest that selecting the right pretraining strategy, especially with SSL, can be pivotal for improving diagnostic accuracy of artificial intelligence in medical imaging., Conclusions: By demonstrating the promise of SSL in chest radiograph analysis, we underline a transformative shift towards more efficient and accurate AI models in medical imaging., Relevance Statement: Self-supervised learning highlights a paradigm shift towards the enhancement of AI-driven accuracy and efficiency in medical imaging. Given its promise, the broader application of self-supervised learning in medical imaging calls for deeper exploration, particularly in contexts where comprehensive annotated datasets are limited., (© 2024. The Author(s).)

Background: Digitized histopathological tissue slides and genomics profiling data are available for many patients with solid tumors. In the last 5 years, Deep Learning (DL) has been broadly used to extract clinically actionable information and biological knowledge from pathology slides and genomic data in cancer. In addition, a number of recent studies have introduced multimodal DL models designed to simultaneously process both images from pathology slides and genomic data as inputs. By comparing patterns from one data modality with those in another, multimodal DL models are capable of achieving higher performance compared to their unimodal counterparts. However, the application of these methodologies across various tumor entities and clinical scenarios lacks consistency., Methods: Here, we present a systematic survey of the academic literature from 2010 to November 2023, aiming to quantify the application of DL for pathology, genomics, and the combined use of both data types. After filtering 3048 publications, our search identified 534 relevant articles which then were evaluated by basic (diagnosis, grading, subtyping) and advanced (mutation, drug response and survival prediction) application types, publication year and addressed cancer tissue., Results: Our analysis reveals a predominant application of DL in pathology compared to genomics. However, there is a notable surge in DL incorporation within both domains. Furthermore, while DL applied to pathology primarily targets the identification of histology-specific patterns in individual tissues, DL in genomics is more commonly used in a pan-cancer context. Multimodal DL, on the contrary, remains a niche topic, evidenced by a limited number of publications, primarily focusing on prognosis predictions., Conclusion: In summary, our quantitative analysis indicates that DL not only has a well-established role in histopathology but is also being successfully integrated into both genomic and multimodal applications. In addition, there is considerable potential in multimodal DL for harnessing further advanced tasks, such as predicting drug response. Nevertheless, this review also underlines the need for further research to bridge the existing gaps in these fields., (© 2024. The Author(s).)

Artificial intelligence (AI) has a multitude of applications in cancer research and oncology. However, the training of AI systems is impeded by the limited availability of large datasets due to data protection requirements and other regulatory obstacles. Federated and swarm learning represent possible solutions to this problem by collaboratively training AI models while avoiding data transfer. However, in these decentralized methods, weight updates are still transferred to the aggregation server for merging the models. This leaves the possibility for a breach of data privacy, for example by model inversion or membership inference attacks by untrusted servers. Somewhat-homomorphically-encrypted federated learning (SHEFL) is a solution to this problem because only encrypted weights are transferred, and model updates are performed in the encrypted space. Here, we demonstrate the first successful implementation of SHEFL in a range of clinically relevant tasks in cancer image analysis on multicentric datasets in radiology and histopathology. We show that SHEFL enables the training of AI models which outperform locally trained models and perform on par with models which are centrally trained. In the future, SHEFL can enable multiple institutions to co-train AI models without forsaking data governance and without ever transmitting any decryptable data to untrusted servers., Competing Interests: Declaration of Competing Interest The Authors declare no competing financial or non-financial interests. For transparency, we provide the following information: JNK declares consulting services for Owkin, France, DoMore Diagnostics, Norway, Panakeia, UK, Scailyte, Switzerland, Cancilico, Germany, Mindpeak, Germany, and Histofy, UK; furthermore he holds shares in StratifAI GmbH, Germany, and has received honoraria for lectures by AstraZeneca, Bayer, Eisai, MSD, BMS, Roche, Pfizer and Fresenius. DT holds shares in StraifAI GmbH, Germany and received honoraria for lectures by Bayer. PQ and NW declare research funding from Roche and PQ consulting and speaker services for Roche. MST has recently received honoraria for advisory work in relation to the following companies: Incyte, MindPeak, MSD, BMS and Sonrai; these are all unrelated to this work. No other potential conflicts of interest are reported by any of the authors. The authors received advice from NVIDIA when performing this study, but NVIDIA did not have any role in study design, conducting the experiments, interpretation of the results or decision to submit for publication., (Copyright © 2023 The Authors. Published by Elsevier B.V. All rights reserved.)

Until recently the application of artificial intelligence (AI) in precision oncology was confined to activities in drug development and had limited impact on the personalisation of therapy. Now, a number of approaches have been proposed for the personalisation of drug and cell therapies with AI applied to therapy design, planning and delivery at the patient's bedside. Some drug and cell-based therapies are already tuneable to the individual to optimise efficacy, to reduce toxicity, to adapt the dosing regime, to design combination therapy approaches and, preclinically, even to personalise the receptor design of cell therapies. Developments in AI-based healthcare are accelerating through the adoption of foundation models, and generalist medical AI models have been proposed. The application of these approaches in therapy design is already being explored and realistic short-term advances include the application to the personalised design and delivery of drugs and cell therapies. With this pace of development, the limiting step to adoption will likely be the capacity and appropriateness of regulatory frameworks. This article explores emerging concepts and new ideas for the regulation of AI-enabled personalised cancer therapies in the context of existing and in development governance frameworks., (© 2024. The Author(s).)

Programmed death-ligand 1 (PD-L1) IHC is the most commonly used biomarker for immunotherapy response. However, quantification of PD-L1 status in pathology slides is challenging. Neither manual quantification nor a computer-based mimicking of manual readouts is perfectly reproducible, and the predictive performance of both approaches regarding immunotherapy response is limited. In this study, we developed a deep learning (DL) method to predict PD-L1 status directly from raw IHC image data, without explicit intermediary steps such as cell detection or pigment quantification. We trained the weakly supervised model on PD-L1-stained slides from the non-small cell lung cancer (NSCLC)-Memorial Sloan Kettering (MSK) cohort (N = 233) and validated it on the pan-cancer-Vall d'Hebron Institute of Oncology (VHIO) cohort (N = 108). We also investigated the performance of the model to predict response to immune checkpoint inhibitors (ICI) in terms of progression-free survival. In the pan-cancer-VHIO cohort, the performance was compared with tumor proportion score (TPS) and combined positive score (CPS). The DL model showed good performance in predicting PD-L1 expression (TPS ≥ 1%) in both NSCLC-MSK and pan-cancer-VHIO cohort (AUC 0.88 ± 0.06 and 0.80 ± 0.03, respectively). The predicted PD-L1 status showed an improved association with response to ICIs [HR: 1.5 (95% confidence interval: 1-2.3), P = 0.049] compared with TPS [HR: 1.4 (0.96-2.2), P = 0.082] and CPS [HR: 1.2 (0.79-1.9), P = 0.386]. Notably, our explainability analysis showed that the model does not just look at the amount of brown pigment in the IHC slides, but also considers morphologic factors such as lymphocyte conglomerates. Overall, end-to-end weakly supervised DL shows potential for improving patient stratification for cancer immunotherapy by analyzing PD-L1 IHC, holistically integrating morphology and PD-L1 staining intensity., Significance: The weakly supervised DL model to predict PD-L1 status from raw IHC data, integrating tumor staining intensity and morphology, enables enhanced patient stratification in cancer immunotherapy compared with traditional pathologist assessment., (© 2024 The Authors; Published by the American Association for Cancer Research.)

Background: Precise prognosis prediction in patients with colorectal cancer (ie, forecasting survival) is pivotal for individualised treatment and care. Histopathological tissue slides of colorectal cancer specimens contain rich prognostically relevant information. However, existing studies do not have multicentre external validation with real-world sample processing protocols, and algorithms are not yet widely used in clinical routine., Methods: In this retrospective, multicentre study, we collected tissue samples from four groups of patients with resected colorectal cancer from Australia, Germany, and the USA. We developed and externally validated a deep learning-based prognostic-stratification system for automatic prediction of overall and cancer-specific survival in patients with resected colorectal cancer. We used the model-predicted risk scores to stratify patients into different risk groups and compared survival outcomes between these groups. Additionally, we evaluated the prognostic value of these risk groups after adjusting for established prognostic variables., Findings: We trained and validated our model on a total of 4428 patients. We found that patients could be divided into high-risk and low-risk groups on the basis of the deep learning-based risk score. On the internal test set, the group with a high-risk score had a worse prognosis than the group with a low-risk score, as reflected by a hazard ratio (HR) of 4·50 (95% CI 3·33-6·09) for overall survival and 8·35 (5·06-13·78) for disease-specific survival (DSS). We found consistent performance across three large external test sets. In a test set of 1395 patients, the high-risk group had a lower DSS than the low-risk group, with an HR of 3·08 (2·44-3·89). In two additional test sets, the HRs for DSS were 2·23 (1·23-4·04) and 3·07 (1·78-5·3). We showed that the prognostic value of the deep learning-based risk score is independent of established clinical risk factors., Interpretation: Our findings indicate that attention-based self-supervised deep learning can robustly offer a prognosis on clinical outcomes in patients with colorectal cancer, generalising across different populations and serving as a potentially new prognostic tool in clinical decision making for colorectal cancer management. We release all source codes and trained models under an open-source licence, allowing other researchers to reuse and build upon our work., Funding: The German Federal Ministry of Health, the Max-Eder-Programme of German Cancer Aid, the German Federal Ministry of Education and Research, the German Academic Exchange Service, and the EU., Competing Interests: Declaration of interests JNK has received consulting fees from Owkin; DoMore Diagnostics; Panakeia; and Histofy; furthermore, JNK holds shares and holds a leadership role in StratifAI and has received honoraria for lectures by Bayer, Eisai, Merck Sharp & Dohme (MSD), Bristol-Myers Squibb (BMS), Roche, Pfizer, and Fresenius and has participated on a Data Safety Monitoring Board or Advisory Board for Bayer, Eisai, MSD, BMS, Roche, and Pfizer. PQ and NPW declare research funding from Roche and PQ consulting and speaker services for Roche. JJ declares consulting fees from WHO (Development of Digital Health Solution) and CARE International, Papua New Guinea (Development of Survey Instruments). SF declares grants and contracts from Bundesministerium für Bildung und Forschung, Deutsche Forschungsgemeinschaft, and German Cancer Aid, and payment or honoraria from BMS, MSD, European Society for Medical Oncology, and Deutsche Gesellschaft für Pathologie. DT holds shares in StratifAI. WM is a shareholder of Gemeinschaftspraxis Pathologie Starnberg, a private pathology practice in Germany. All other authors declare no competing interests., (Copyright © 2024 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license. Published by Elsevier Ltd.. All rights reserved.)

Primary liver cancer arises either from hepatocytic or biliary lineage cells, giving rise to hepatocellular carcinoma (HCC) or intrahepatic cholangiocarcinoma (ICCA). Combined hepatocellular- cholangiocarcinomas (cHCC-CCA) exhibit equivocal or mixed features of both, causing diagnostic uncertainty and difficulty in determining proper management. Here, we perform a comprehensive deep learning-based phenotyping of multiple cohorts of patients. We show that deep learning can reproduce the diagnosis of HCC vs. CCA with a high performance. We analyze a series of 405 cHCC-CCA patients and demonstrate that the model can reclassify the tumors as HCC or ICCA, and that the predictions are consistent with clinical outcomes, genetic alterations and in situ spatial gene expression profiling. This type of approach could improve treatment decisions and ultimately clinical outcome for patients with rare and biphenotypic cancers such as cHCC-CCA., (© 2023. The Author(s).)

Breast cancer prognosis and management for both men and women are reliant upon estrogen receptor alpha (ERα) and progesterone receptor (PR) expression to inform therapy. Previous studies have shown that there are sex-specific binding characteristics of ERα and PR in breast cancer and, counterintuitively, ERα expression is more common in male than female breast cancer. We hypothesized that these differences could have morphological manifestations that are undetectable to human observers but could be elucidated computationally. To investigate this, we trained attention-based multiple instance learning prediction models for ERα and PR using H&E-stained images of female breast cancer from the Cancer Genome Atlas (TCGA) (n = 1085) and deployed them on external female (n = 192) and male breast cancer images (n = 245). Both targets were predicted in the internal (AUROC for ERα prediction: 0.86 ± 0.02, p < 0.001; AUROC for PR prediction = 0.76 ± 0.03, p < 0.001) and external female cohorts (AUROC for ERα prediction: 0.78 ± 0.03, p < 0.001; AUROC for PR prediction = 0.80 ± 0.04, p < 0.001) but not the male cohort (AUROC for ERα prediction: 0.66 ± 0.14, p = 0.43; AUROC for PR prediction = 0.63 ± 0.04, p = 0.05). This suggests that subtle morphological differences invisible upon visual inspection may exist between the sexes, supporting previous immunohistochemical, genomic, and transcriptomic analyses., (© 2023. The Author(s).)

Mismatch Repair Deficiency (dMMR)/Microsatellite Instability (MSI) is a key biomarker in colorectal cancer (CRC). Universal screening of CRC patients for MSI status is now recommended, but contributes to increased workload for pathologists and delayed therapeutic decisions. Deep learning has the potential to ease dMMR/MSI testing and accelerate oncologist decision making in clinical practice, yet no comprehensive validation of a clinically approved tool has been conducted. We developed MSIntuit, a clinically approved artificial intelligence (AI) based pre-screening tool for MSI detection from haematoxylin-eosin (H&E) stained slides. After training on samples from The Cancer Genome Atlas (TCGA), a blind validation is performed on an independent dataset of 600 consecutive CRC patients. Inter-scanner reliability is studied by digitising each slide using two different scanners. MSIntuit yields a sensitivity of 0.96-0.98, a specificity of 0.47-0.46, and an excellent inter-scanner agreement (Cohen's κ: 0.82). By reaching high sensitivity comparable to gold standard methods while ruling out almost half of the non-MSI population, we show that MSIntuit can effectively serve as a pre-screening tool to alleviate MSI testing burden in clinical practice., (© 2023. Springer Nature Limited.)

Background & Aims: Diagnosis of adenocarcinoma in the liver is a frequent scenario in routine pathology and has a critical impact on clinical decision making. However, rendering a correct diagnosis can be challenging, and often requires the integration of clinical, radiologic, and immunohistochemical information. We present a deep learning model (HEPNET) to distinguish intrahepatic cholangiocarcinoma from colorectal liver metastasis, as the most frequent primary and secondary forms of liver adenocarcinoma, with clinical grade accuracy using H&E-stained whole-slide images., Methods: HEPNET was trained on 714,589 image tiles from 456 patients who were randomly selected in a stratified manner from a pool of 571 patients who underwent surgical resection or biopsy at Heidelberg University Hospital. Model performance was evaluated on a hold-out internal test set comprising 115 patients and externally validated on 159 patients recruited at Mainz University Hospital., Results: On the hold-out internal test set, HEPNET achieved an area under the receiver operating characteristic curve of 0.994 (95% CI, 0.989-1.000) and an accuracy of 96.522% (95% CI, 94.521%-98.694%) at the patient level. Validation on the external test set yielded an area under the receiver operating characteristic curve of 0.997 (95% CI, 0.995-1.000), corresponding to an accuracy of 98.113% (95% CI, 96.907%-100.000%). HEPNET surpassed the performance of 6 pathology experts with different levels of experience in a reader study of 50 patients (P = .0005), boosted the performance of resident pathologists to the level of senior pathologists, and reduced potential downstream analyses., Conclusions: We provided a ready-to-use tool with clinical grade performance that may facilitate routine pathology by rendering a definitive diagnosis and guiding ancillary testing. The incorporation of HEPNET into pathology laboratories may optimize the diagnostic workflow, complemented by test-related labor and cost savings., (Copyright © 2023 AGA Institute. Published by Elsevier Inc. All rights reserved.)

Large language models (LLMs) are artificial intelligence (AI) tools specifically trained to process and generate text. LLMs attracted substantial public attention after OpenAI's ChatGPT was made publicly available in November 2022. LLMs can often answer questions, summarize, paraphrase and translate text on a level that is nearly indistinguishable from human capabilities. The possibility to actively interact with models like ChatGPT makes LLMs attractive tools in various fields, including medicine. While these models have the potential to democratize medical knowledge and facilitate access to healthcare, they could equally distribute misinformation and exacerbate scientific misconduct due to a lack of accountability and transparency. In this article, we provide a systematic and comprehensive overview of the potentials and limitations of LLMs in clinical practice, medical research and medical education., (© 2023. Springer Nature Limited.)

Background: Artificial intelligence (AI) is influencing our society on many levels and has broad implications for the future practice of hematology and oncology. However, for many medical professionals and researchers, it often remains unclear what AI can and cannot do, and what are promising areas for a sensible application of AI in hematology and oncology. Finally, the limits and perils of using AI in oncology are not obvious to many healthcare professionals., Methods: In this article, we provide an expert-based consensus statement by the joint Working Group on "Artificial Intelligence in Hematology and Oncology" by the German Society of Hematology and Oncology (DGHO), the German Association for Medical Informatics, Biometry and Epidemiology (GMDS), and the Special Interest Group Digital Health of the German Informatics Society (GI). We provide a conceptual framework for AI in hematology and oncology., Results: First, we propose a technological definition, which we deliberately set in a narrow frame to mainly include the technical developments of the last ten years. Second, we present a taxonomy of clinically relevant AI systems, structured according to the type of clinical data they are used to analyze. Third, we show an overview of potential applications, including clinical, research, and educational environments with a focus on hematology and oncology., Conclusion: Thus, this article provides a point of reference for hematologists and oncologists, and at the same time sets forth a framework for the further development and clinical deployment of AI in hematology and oncology in the future., (© 2023. The Author(s).)

When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a transformer-based neural network as a novel AI architecture that integrates multimodal patient data, i.e., imaging data (chest radiographs) and non-imaging data (clinical data). We evaluate the performance of our model in a retrospective study with 6,125 patients in intensive care. We show that the combined model (area under the receiver operating characteristic curve [AUROC] of 0.863) is superior to the radiographs-only model (AUROC = 0.811, p < 0.001) and the clinical data-only model (AUROC = 0.785, p < 0.001) when tasked with predicting in-hospital survival per patient. Furthermore, we demonstrate that our proposed model is robust in cases where not all (clinical) data points are available., (© 2023. The Author(s).)

Pathologists worldwide are facing remarkable challenges with increasing workloads and lack of time to provide consistently high-quality patient care. The application of artificial intelligence (AI) to digital whole-slide images has the potential of democratizing the access to expert pathology and affordable biomarkers by supporting pathologists in the provision of timely and accurate diagnosis as well as supporting oncologists by directly extracting prognostic and predictive biomarkers from tissue slides. The long-awaited adoption of AI in pathology, however, has not materialized, and the transformation of pathology is happening at a much slower pace than that observed in other fields (eg, radiology). Here, we provide a critical summary of the developments in digital and computational pathology in the last 10 years, outline key hurdles and ways to overcome them, and provide a perspective for AI-supported precision oncology in the future., (© The Author(s) 2023. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com.)

The histopathological phenotype of tumors reflects the underlying genetic makeup. Deep learning can predict genetic alterations from pathology slides, but it is unclear how well these predictions generalize to external datasets. We performed a systematic study on Deep-Learning-based prediction of genetic alterations from histology, using two large datasets of multiple tumor types. We show that an analysis pipeline that integrates self-supervised feature extraction and attention-based multiple instance learning achieves a robust predictability and generalizability., (© 2023. The Author(s).)

Aims: One of the most important complications of heart transplantation is organ rejection, which is diagnosed on endomyocardial biopsies by pathologists. Computer-based systems could assist in the diagnostic process and potentially improve reproducibility. Here, we evaluated the feasibility of using deep learning in predicting the degree of cellular rejection from pathology slides as defined by the International Society for Heart and Lung Transplantation (ISHLT) grading system., Methods and Results: We collected 1079 histopathology slides from 325 patients from three transplant centres in Germany. We trained an attention-based deep neural network to predict rejection in the primary cohort and evaluated its performance using cross-validation and by deploying it to three cohorts. For binary prediction (rejection yes/no), the mean area under the receiver operating curve (AUROC) was 0.849 in the cross-validated experiment and 0.734, 0.729, and 0.716 in external validation cohorts. For a prediction of the ISHLT grade (0R, 1R, 2/3R), AUROCs were 0.835, 0.633, and 0.905 in the cross-validated experiment and 0.764, 0.597, and 0.913; 0.631, 0.633, and 0.682; and 0.722, 0.601, and 0.805 in the validation cohorts, respectively. The predictions of the artificial intelligence model were interpretable by human experts and highlighted plausible morphological patterns., Conclusion: We conclude that artificial intelligence can detect patterns of cellular transplant rejection in routine pathology, even when trained on small cohorts., Competing Interests: Conflict of interest: J.N.K. declares consulting services for Owkin, France, and Panakeia, UK, as well as reimbursement for scientific talks by MSD, Eisai, and Fresenius. D.W. declares consulting services and honorary talks for Abiomed, AstraZeneca, Bayer, Berlin-Chemie, Novartis, Medtronic. CM declares honorary talks for AstraZeneca, Novartis, Heinen&Loewenstein, Boehringer Ingelheim/Lilly, Bayer, Pfizer, Sanofi, Aventis, Apontis, and Abbott and meeting support from AstraZeneca, Novartis, and Boehringer Ingelheim/Lilly. T.L. declares consulting fees from AstraZeneca, BMS, EISAI, Incyte, MSD, Roche, and HepaRegeniX and honorary talks and travel support from Abbvie and Gilead. The other authors do not have anything to disclose., (© The Author(s) 2023. Published by Oxford University Press on behalf of the European Society of Cardiology.)

Aims: Immune checkpoint inhibitor (ICI) therapy has become a viable treatment strategy in bladder cancer. However, treatment responses vary, and improved biomarkers are needed. Crucially, the characteristics of immune cells remain understudied especially in squamous differentiated bladder cancer (sq-BLCA). Here, we quantitatively analysed the tumour-immune phenotypes of sq-BLCA and correlated them with PD-L1 expression and FGFR3 mutation status., Methods: Tissue microarrays (TMA) of n = 68 non-schistosomiasis associated pure squamous cell carcinoma (SCC) and n = 46 mixed urothelial carcinoma with squamous differentiation (MIX) were subjected to immunohistochemistry for CD3, CD4, CD8, CD56, CD68, CD79A, CD163, Ki67, perforin and chloroacetate esterase staining. Quantitative image evaluation was performed via digital image analysis., Results: Immune infiltration was generally higher in stroma than in tumour regions. B-cells (CD79A) were almost exclusively found in stromal areas (sTILs), T-lymphocytes and macrophages were also present in tumour cell areas (iTILs), while natural killer cells (CD56) were nearly missing in any area. Tumour-immune phenotype distribution differed depending on the immune cell subset, however, hot tumour-immune phenotypes (high density of immune cells in tumour areas) were frequently found for CD8 + T-cells (33%), especially perforin + lymphocytes (52.2%), and CD68 + macrophages (37.6%). Perforin + CD8 lymphocytes predicted improved overall survival in sq-BLCA while high PD-L1 expression (CPS ≥ 10) was significantly associated with higher CD3 + , CD8 + and CD163 + immune cell density and high Ki67 (density) of tumour cells. Furthermore, PD-L1 expression was positively associated with CD3 + /CD4 + , CD3 + /CD8 + and CD68 + /CD163 + hot tumour-immune phenotypes. FGFR3 mutation status was inversely associated with CD8 + , perforin + and CD79A + lymphocyte density., Conclusions: Computer-based image analysis is an efficient tool to analyse immune topographies in squamous bladder cancer. Hot tumour-immune phenotypes with strong PD-L1 expression might pose a promising subgroup for clinically successful ICI therapy in squamous bladder cancer and warrant further investigation., (© 2023. The Author(s).)

Immunotherapy by immune checkpoint inhibitors has become a standard treatment strategy for many types of solid tumors. However, the majority of patients with cancer will not respond, and predicting response to this therapy is still a challenge. Artificial intelligence (AI) methods can extract meaningful information from complex data, such as image data. In clinical routine, radiology or histopathology images are ubiquitously available. AI has been used to predict the response to immunotherapy from radiology or histopathology images, either directly or indirectly via surrogate markers. While none of these methods are currently used in clinical routine, academic and commercial developments are pointing toward potential clinical adoption in the near future. Here, we summarize the state of the art in AI-based image biomarkers for immunotherapy response based on radiology and histopathology images. We point out limitations, caveats, and pitfalls, including biases, generalizability, and explainability, which are relevant for researchers and health care providers alike, and outline key clinical use cases of this new class of predictive biomarkers., (©2022 American Association for Cancer Research.)

Background & Aims: Combined hepatocellular-cholangiocarcinoma (cHCC-CCA) is a rare primary liver cancer (PLC) associated with a poor prognosis. Given the challenges in its identification and its clinical implications, biomarkers are critically needed. We aimed to investigate the diagnostic and prognostic value of the immunohistochemical expression of Nestin, a progenitor cell marker, in a large multicentric series of PLCs., Methods: We collected 461 cHCC-CCA samples from 32 different clinical centers. Control cases included 368 hepatocellular carcinomas (HCCs) and 221 intrahepatic cholangiocarcinomas (iCCAs). Nestin immunohistochemistry was performed on whole tumor sections. Diagnostic and prognostic performances of Nestin expression were determined using receiver-operating characteristic curves and Cox regression modeling., Results: Nestin was able to distinguish cHCC-CCA from HCC with AUCs of 0.85 and 0.86 on surgical and biopsy samples, respectively. Performance was lower for the distinction of cHCC-CCA from iCCA (AUCs of 0.59 and 0.60). Nestin, however, showed a high prognostic value, allowing identification of the subset of cHCC-CCA ("Nestin High", >30% neoplastic cells with positive staining) associated with the worst clinical outcome (shorter disease-free and overall survival) after surgical resection and liver transplantation, as well as when assessment was performed on biopsies., Conclusion: We show in different clinical settings that Nestin has diagnostic value and that it is a useful biomarker to identify the subset of cHCC-CCA associated with the worst clinical outcome. Nestin immunohistochemistry may be used to refine risk stratification and improve treatment allocation for patients with this highly aggressive malignancy., Lay Summary: There are different types of primary liver cancers (i.e. cancers that originate in the liver). Accurately identifying a specific subtype of primary liver cancer (and determining its associated prognosis) is important as it can have a major impact on treatment allocation. Herein, we show that a protein called Nestin could be used to refine risk stratification and improve treatment allocation for patients with combined hepatocellular carcinoma, a rare but highly aggressive subtype of primary liver cancer., Competing Interests: Conflict of interest JC consults for Crosscope, KB is Crosscope Chief Technology Officer, JS is Crosscope Chief Executive Officer. Please refer to the accompanying ICMJE disclosure forms for further details., (Copyright © 2022 European Association for the Study of the Liver. Published by Elsevier B.V. All rights reserved.)

Artificial intelligence (AI) methods have multiplied our capabilities to extract quantitative information from digital histopathology images. AI is expected to reduce workload for human experts, improve the objectivity and consistency of pathology reports, and have a clinical impact by extracting hidden information from routinely available data. Here, we describe how AI can be used to predict cancer outcome, treatment response, genetic alterations and gene expression from digitized histopathology slides. We summarize the underlying technologies and emerging approaches, noting limitations, including the need for data sharing and standards. Finally, we discuss the broader implications of AI in cancer research and oncology., (© 2022. Springer Nature America, Inc.)

Artificial intelligence (AI) is currently transforming all aspects of our daily life, including the practice of medicine. Artificial neural networks are a key method of AI. They can very effectively detect subtle patterns in imaging data and speech or text data. This is highly relevant for the practice of gastroenterology. Here, we summarize the state of the art in AI in gastroenterology and outline major clinical applications. Our focus is on AI-based analysis of endoscopy images, non-invasive imaging and histology images. In these applications, AI can support human pattern recognition. Beyond detection and classification of pathological findings, AI can predict clinical outcome from subtle image features., Competing Interests: Die Autorinnen/Autoren geben an, dass kein Interessenkonflikt besteht., (Thieme. All rights reserved.)