Search

IntroductionThis study is a retrospective evaluation of the performance of deep learning models that were developed for the detection of COVID-19 from chest x-rays, undertaken with the goal of assessing the suitability of such systems as clinical decision support tools.MethodsModels were trained on the National COVID-19 Chest Imaging Database (NCCID), a UK-wide multi-centre dataset from 26 different NHS hospitals and evaluated on independent multi-national clinical datasets. The evaluation considers clinical and technical contributors to model error and potential model bias. Model predictions are examined for spurious feature correlations using techniques for explainable prediction.ResultsModels performed adequately on NHS populations, with performance comparable to radiologists, but generalised poorly to international populations. Models performed better in males than females, and performance varied across age groups. Alarmingly, models routinely failed when applied to complex clinical cases with confounding pathologies and when applied to radiologist defined “mild” cases.DiscussionThis comprehensive benchmarking study examines the pitfalls in current practices that have led to impractical model development. Key findings highlight the need for clinician involvement at all stages of model development, from data curation and label definition, to model evaluation, to ensure that all clinical factors and disease features are appropriately considered during model design. This is imperative to ensure automated approaches developed for disease detection are fit-for-purpose in a clinical setting.

Abstract This study evaluates the quality of published research using artificial intelligence (AI) for ovarian cancer diagnosis or prognosis using histopathology data. A systematic search of PubMed, Scopus, Web of Science, Cochrane CENTRAL, and WHO-ICTRP was conducted up to May 19, 2023. Inclusion criteria required that AI was used for prognostic or diagnostic inferences in human ovarian cancer histopathology images. Risk of bias was assessed using PROBAST. Information about each model was tabulated and summary statistics were reported. The study was registered on PROSPERO (CRD42022334730) and PRISMA 2020 reporting guidelines were followed. Searches identified 1573 records, of which 45 were eligible for inclusion. These studies contained 80 models of interest, including 37 diagnostic models, 22 prognostic models, and 21 other diagnostically relevant models. Common tasks included treatment response prediction (11/80), malignancy status classification (10/80), stain quantification (9/80), and histological subtyping (7/80). Models were developed using 1–1375 histopathology slides from 1–776 ovarian cancer patients. A high or unclear risk of bias was found in all studies, most frequently due to limited analysis and incomplete reporting regarding participant recruitment. Limited research has been conducted on the application of AI to histopathology images for diagnostic or prognostic purposes in ovarian cancer, and none of the models have been demonstrated to be ready for real-world implementation. Key aspects to accelerate clinical translation include transparent and comprehensive reporting of data provenance and modelling approaches, and improved quantitative evaluation using cross-validation and external validations. This work was funded by the Engineering and Physical Sciences Research Council.

Introduction More people are living with and beyond a cancer diagnosis. There is limited understanding of the long-term effects of cancer and cancer treatment on quality of life and personal and household finances when compared to people without cancer. In a separate protocol we have proposed to link de-identified data from electronic primary care and hospital records for a large population of cancer survivors and matched controls. In this current protocol, we propose the linkage of Patient Reported Outcomes Measures data to the above data for a subset of this population. The aim of this study is to investigate the full impact of living with and beyond a cancer diagnosis compared to age and gender matched controls. A secondary aim is to test the feasibility of the collection of Patient Reported Outcomes Measures (PROMS) data and the linkage procedures of the PROMs data to electronic health records data. Materials and methods This is a cross-sectional study, aiming to recruit participants treated at the Leeds Teaching Hospitals National Health Service Trust. Eligible patients will be cancer survivors at around 5 years post-diagnosis (breast, colorectal and ovarian cancer) and non-cancer patient matched controls attending dermatology out-patient clinics. They will be identified by running a query on the Leeds Teaching Hospitals Trust patient records system. Approximately 6000 patients (2000 cases and 4000 controls) will be invited to participate via post. Participants will be invited to complete PROMs assessing factors such as quality of life and finances, which can be completed on paper or online (surveys includes established instruments, and bespoke instruments (demographics, financial costs). This PROMs data will then be linked to routinely collected de-identified data from patient’s electronic primary care and hospital records. Discussion This innovative work aims to create a truly ‘comprehensive patient record’ to provide a broad picture of what happens to cancer patients across their cancer pathway, and the long-term impact of cancer treatment. Comparisons can be made between the cases and controls, to identify the aspects of life that has had the greatest impact following a cancer diagnosis. The feasibility of linking PROMs data to electronic health records can also be assessed. This work can inform future support offered to people living with and beyond a cancer diagnosis, clinical practice, and future research methodologies.

Background The use of linked healthcare data in research has the potential to make major contributions to knowledge generation and service improvement. However, using healthcare data for secondary purposes raises legal and ethical concerns relating to confidentiality, privacy and data protection rights. Using a linkage and anonymisation approach that processes data lawfully and in line with ethical best practice to create an anonymous (non-personal) dataset can address these concerns, yet there is no set approach for defining all of the steps involved in such data flow end-to-end. We aimed to define such an approach with clear steps for dataset creation, and to describe its utilisation in a case study linking healthcare data. Methods We developed a data flow protocol that generates pseudonymous datasets that can be reversibly linked, or irreversibly linked to form an anonymous research dataset. It was designed and implemented by the Comprehensive Patient Records (CPR) study in Leeds, UK. Results We defined a clear approach that received ethico-legal approval for use in creating an anonymous research dataset. Our approach used individual-level linkage through a mechanism that is not computer-intensive and was rendered irreversible to both data providers and processors. We successfully applied it in the CPR study to hospital and general practice and community electronic health record data from two providers, along with patient reported outcomes, for 365,193 patients. The resultant anonymous research dataset is available via DATA-CAN, the Health Data Research Hub for Cancer in the UK. Conclusions Through ethical, legal and academic review, we believe that we contribute a defined approach that represents a framework that exceeds current minimum standards for effective pseudonymisation and anonymisation. This paper describes our methods and provides supporting information to facilitate the use of this approach in research.

Objectives: To characterize treatment patterns and survival outcomes for patients with locally advanced or metastatic malignancy of the urothelial tract during a period immediately preceding the widespread use of immune checkpoint inhibitors in the UK.Patients and Methods: We retrospectively examined the electronic case notes of patients attending the Leeds Cancer Center, UK with locally advanced or metastatic urothelial carcinoma, receiving chemotherapy between January 2003 and March 2017. Patient characteristics, treatment patterns, and outcomes were collected. Summary and descriptive statistics were calculated for categorical and continuous variables as appropriate. The Kaplan–Meier method was used to estimate median survival and Cox regression proportional hazards model was used to explore relationships between clinical variables and outcome.Results: Two hundred and sixteen patients made up the study cohort, with a median age of 66 years (range: 35–83) and 72.7% being male. First-line treatment consisted of either a cisplatin- (44%) or carboplatin-based regimen (48%) in the majority of patients. Twenty seven percent of patients received a second-line of treatment (most commonly single-agent paclitaxel) following a first-line platinum containing regimen. Grade 4 neutropenia was observed in 19 and 27% of those treated with a first-line cisplatin- and carboplatin-based regimen, respectively. The median overall survival (mOS) of the study cohort was estimated to be 16.2 months (IQR: 10.6–28.3 months). Receipt by patients of cisplatin-based chemotherapy was associated with a longer mOS and this association persisted when survival analysis was adjusted for age, sex, performance status and presence of distant metastases.Conclusions: This study provides a useful benchmark for outcomes achieved in a real-world setting for patients with locally advanced or metastatic UC treated with chemotherapy in the immediate pre-immunotherapy era.