Search

Rigorous evaluation of artificial intelligence (AI) systems for image classification is essential before deployment into health-care settings, such as screening programmes, so that adoption is effective and safe. A key step in the evaluation process is the external validation of diagnostic performance using a test set of images. We conducted a rapid literature review on methods to develop test sets, published from 2012 to 2020, in English. Using thematic analysis, we mapped themes and coded the principles using the Population, Intervention, and Comparator or Reference standard, Outcome, and Study design framework. A group of screening and AI experts assessed the evidence-based principles for completeness and provided further considerations. From the final 15 principles recommended here, five affect population, one intervention, two comparator, one reference standard, and one both reference standard and comparator. Finally, four are appliable to outcome and one to study design. Principles from the literature were useful to address biases from AI; however, they did not account for screening specific biases, which we now incorporate. The principles set out here should be used to support the development and use of test sets for studies that assess the accuracy of AI within screening programmes, to ensure they are fit for purpose and minimise bias.

Artificial intelligence (AI) could have the potential to accurately classify mammograms according to the presence or absence of radiological signs of breast cancer, replacing or supplementing human readers (radiologists). The UK National Screening Committee's assessments of the use of AI systems to examine screening mammograms continues to focus on maximising benefits and minimising harms to women screened, when deciding whether to recommend the implementation of AI into the Breast Screening Programme in the UK. Maintaining or improving programme specificity is important to minimise anxiety from false positive results. When considering cancer detection, AI test sensitivity alone is not sufficiently informative, and additional information on the spectrum of disease detected and interval cancers is crucial to better understand the benefits and harms of screening. Although large retrospective studies might provide useful evidence by directly comparing test accuracy and spectrum of disease detected between different AI systems and by population subgroup, most retrospective studies are biased due to differential verification (ie, the use of different reference standards to verify the target condition among study participants). Enriched, multiple-reader, multiple-case, test set laboratory studies are also biased due to the laboratory effect (ie, radiologists' performance in retrospective, laboratory, observer studies is substantially different to their performance in a clinical environment). Therefore, assessment of the effect of incorporating any AI system into the breast screening pathway in prospective studies is required as it will provide key evidence for the effect of the interaction of medical staff with AI, and the impact on women's outcomes.

Purpose To determine classification criteria for Vogt-Koyanagi-Harada (VKH) disease Design Machine learning of cases with VKH disease and 5 other panuveitides. Methods Cases of panuveitides were collected in an informatics-designed preliminary database, and a final database was constructed of cases achieving supermajority agreement on the diagnosis, using formal consensus techniques. Cases were split into a training set and a validation set. Machine learning using multinomial logistic regression was used on the training set to determine a parsimonious set of criteria that minimized the misclassification rate among the panuveitides. The resulting criteria were evaluated on the validation set. Results One thousand twelve cases of panuveitides, including 156 cases of early-stage VKH and 103 cases of late-stage VKH, were evaluated. Overall accuracy for panuveitides was 96.3% in the training set and 94.0% in the validation set (95% confidence interval 89.0, 96.8). Key criteria for early-stage VKH included: 1) exudative retinal detachment with characteristic appearance on fluorescein angiogram or optical coherence tomography or 2) panuveitis with ≥2 of 5 neurologic symptoms/signs. Key criteria for late-stage VKH included history of early-stage VKH and either: 1) sunset glow fundus or 2) uveitis and ≥1 of 3 cutaneous signs. The misclassification rates in the learning and validation sets for early-stage VKH were 8.0% and 7.7%, respectively, and for late-stage VKH 1.0% and 12%, respectively. Conclusions The criteria for VKH had a reasonably low misclassification rate and appeared to perform sufficiently well for use in clinical and translational research.

Purpose To determine classification criteria for sarcoidosis-associated uveitis DESIGN: Machine learning of cases with sarcoid uveitis and 15 other uveitides. Methods Cases of anterior, intermediate, and panuveitides were collected in an informatics-designed preliminary database, and a final database was constructed of cases achieving supermajority agreement on the diagnosis, using formal consensus techniques. Cases were analyzed by anatomic class, and each class was split into a training set and a validation set. Machine learning using multinomial logistic regression was used on the training sets to determine a parsimonious set of criteria that minimized the misclassification rate among the intermediate uveitides. The resulting criteria were evaluated on the validation sets. Results One thousand eighty-three anterior uveitides, 589 intermediate uveitides, and 1012 panuveitides, including 278 cases of sarcoidosis-associated uveitis, were evaluated by machine learning. Key criteria for sarcoidosis-associated uveitis included a compatible uveitic syndrome of any anatomic class and evidence of sarcoidosis, either 1) a tissue biopsy demonstrating non-caseating granulomata or 2) bilateral hilar adenopathy on chest imaging. The overall accuracy of the diagnosis of sarcoidosis-associated uveitis in the validation set was 99.7% (95% confidence interval 98.8, 99.9).The misclassification rates for sarcoidosis-associated uveitis in the training sets were: anterior uveitis 3.2%, intermediate uveitis 2.6%, and panuveitis 1.2%; in the validation sets the misclassification rates were: anterior uveitis 0%, intermediate uveitis 0%, and panuveitis 0%, respectively. Conclusions The criteria for sarcoidosis-associated uveitis had a low misclassification rate and appeared to perform sufficiently well for use in clinical and translational research.

Data-driven digital health technologies have the power to transform health care. If these tools could be sustainably delivered at scale, they might have the potential to provide everyone, everywhere, with equitable access to expert-level care, narrowing the global health and wellbeing gap. Conversely, it is highly possible that these transformative technologies could exacerbate existing health-care inequalities instead. In this Viewpoint, we describe the problem of health data poverty: the inability for individuals, groups, or populations to benefit from a discovery or innovation due to a scarcity of data that are adequately representative. We assert that health data poverty is a threat to global health that could prevent the benefits of data-driven digital health technologies from being more widely realised and might even lead to them causing harm. We argue that the time to act is now to avoid creating a digital health divide that exacerbates existing health-care inequalities and to ensure that no one is left behind in the digital era.

Topic An international, expert-led consensus initiative organized by the Collaborative Ocular Tuberculosis Study (COTS), along with the International Ocular Inflammation Society and the International Uveitis Study Group, systematically developed evidence- and experience-based recommendations for the treatment of tubercular choroiditis. Clinical relevance The diagnosis and management of tubercular uveitis (TBU) pose a significant challenge. Current guidelines and literature are insufficient to guide physicians regarding the initiation of antitubercular therapy (ATT) in patients with TBU. Methods An international expert steering subcommittee of the COTS group identified clinical questions and conducted a systematic review of the published literature on the use of ATT for tubercular choroiditis. Using an interactive online questionnaire, guided by background knowledge from published literature, 81 global experts (including ophthalmologists, pulmonologists, and infectious disease physicians) generated preliminary consensus statements for initiating ATT in tubercular choroiditis, using Oxford levels of medical evidence. In total, 162 statements were identified regarding when to initiate ATT in patients with tubercular serpiginous-like choroiditis, tuberculoma, and tubercular focal or multifocal choroiditis. The COTS group members met in November 2018 to refine these statements by a 2-step modified Delphi process. Results Seventy consensus statements addressed the initiation of ATT in the 3 subtypes of tubercular choroiditis, and in addition, 10 consensus statements were developed regarding the use of adjunctive therapy in tubercular choroiditis. Experts agreed on initiating ATT in tubercular choroiditis in the presence of positive results for any 1 of the positive immunologic tests along with radiologic features suggestive of tuberculosis. For tubercular serpiginous-like choroiditis and tuberculoma, positive results from even 1 positive immunologic test were considered sufficient to recommend ATT, even if there were no radiologic features suggestive of tuberculosis. Discussion Consensus guidelines were developed to guide the initiation of ATT in patients with tubercular choroiditis, based on the published literature, expert opinion, and practical experience, to bridge the gap between clinical need and available medical evidence.

Topic The Collaborative Ocular Tuberculosis Study (COTS), supported by the International Ocular Inflammation Society, International Uveitis Study Group, and Foster Ocular Immunological Society, set up an international, expert-led consensus project to develop evidence- and experience-based guidelines for the management of tubercular uveitis (TBU). Clinical Relevance The absence of international agreement on the use of antitubercular therapy (ATT) in patients with TBU contributes to a significant heterogeneity in the approach to the management of this condition. Methods Consensus statements for the initiation of ATT in TBU were generated using a 2-step modified Delphi technique. In Delphi step 1, a smart web-based survey based on background evidence from published literature was prepared to collect the opinion of 81 international experts on the use of ATT in different clinical scenarios. The survey included 324 questions related to tubercular anterior uveitis (TAU), tubercular intermediate uveitis (TIU), tubercular panuveitis (TPU), and tubercular retinal vasculitis (TRV) administered by the experts, after which the COTS group met in November 2019 for a systematic and critical discussion of the statements in accordance with the second round of the modified Delphi process. Results Forty-four consensus statements on the initiation of ATT in TAU, TIU, TPU, and TRV were obtained, based on ocular phenotypes suggestive of TBU and corroborative evidence of tuberculosis, provided by several combinations of immunologic and radiologic test results. Experts agreed on initiating ATT in recurrent TAU, TIU, TPU, and active TRV depending on the TB endemicity. In the presence of positive results for any 1 of the immunologic tests along with radiologic features suggestive of past evidence of tuberculosis infection. In patients with a first episode of TAU, consensus to initiate ATT was reached only if both immunologic and radiologic test results were positive. Discussion The COTS consensus guidelines were generated based on the evidence from published literature, specialists’ opinions, and logic construction to address the initiation of ATT in TBU. The guidelines also should inform public policy by adding specific types of TBU to the list of conditions that should be treated as tuberculosis.

The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human–AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial., The CONSORT-AI and SPIRIT-AI extensions improve the transparency of clinical trial design and trial protocol reporting for artificial intelligence interventions.

Summary: Background: We undertook a Grand Challenges in Global Eye Health prioritisation exercise to identify the key issues that must be addressed to improve eye health in the context of an ageing population, to eliminate persistent inequities in health-care access, and to mitigate widespread resource limitations. Methods: Drawing on methods used in previous Grand Challenges studies, we used a multi-step recruitment strategy to assemble a diverse panel of individuals from a range of disciplines relevant to global eye health from all regions globally to participate in a three-round, online, Delphi-like, prioritisation process to nominate and rank challenges in global eye health. Through this process, we developed both global and regional priority lists. Findings: Between Sept 1 and Dec 12, 2019, 470 individuals complete round 1 of the process, of whom 336 completed all three rounds (round 2 between Feb 26 and March 18, 2020, and round 3 between April 2 and April 25, 2020) 156 (46%) of 336 were women, 180 (54%) were men. The proportion of participants who worked in each region ranged from 104 (31%) in sub-Saharan Africa to 21 (6%) in central Europe, eastern Europe, and in central Asia. Of 85 unique challenges identified after round 1, 16 challenges were prioritised at the global level; six focused on detection and treatment of conditions (cataract, refractive error, glaucoma, diabetic retinopathy, services for children and screening for early detection), two focused on addressing shortages in human resource capacity, five on other health service and policy factors (including strengthening policies, integration, health information systems, and budget allocation), and three on improving access to care and promoting equity. Interpretation: This list of Grand Challenges serves as a starting point for immediate action by funders to guide investment in research and innovation in eye health. It challenges researchers, clinicians, and policy makers to build collaborations to address specific challenges. Funding: The Queen Elizabeth Diamond Jubilee Trust, Moorfields Eye Charity, National Institute for Health Research Moorfields Biomedical Research Centre, Wellcome Trust, Sightsavers, The Fred Hollows Foundation, The Seva Foundation, British Council for the Prevention of Blindness, and Christian Blind Mission. Translations: For the French, Spanish, Chinese, Portuguese, Arabic and Persian translations of the abstract see Supplementary Materials section.

Summary Background Deep learning has the potential to transform health care; however, substantial expertise is required to train such models. We sought to evaluate the utility of automated deep learning software to develop medical image diagnostic classifiers by health-care professionals with no coding—and no deep learning—expertise. Methods We used five publicly available open-source datasets: retinal fundus images (MESSIDOR); optical coherence tomography (OCT) images (Guangzhou Medical University and Shiley Eye Institute, version 3); images of skin lesions (Human Against Machine [HAM] 10000), and both paediatric and adult chest x-ray (CXR) images (Guangzhou Medical University and Shiley Eye Institute, version 3 and the National Institute of Health [NIH] dataset, respectively) to separately feed into a neural architecture search framework, hosted through Google Cloud AutoML, that automatically developed a deep learning architecture to classify common diseases. Sensitivity (recall), specificity, and positive predictive value (precision) were used to evaluate the diagnostic properties of the models. The discriminative performance was assessed using the area under the precision recall curve (AUPRC). In the case of the deep learning model developed on a subset of the HAM10000 dataset, we did external validation using the Edinburgh Dermofit Library dataset. Findings Diagnostic properties and discriminative performance from internal validations were high in the binary classification tasks (sensitivity 73·3–97·0%; specificity 67–100%; AUPRC 0·87–1·00). In the multiple classification tasks, the diagnostic properties ranged from 38% to 100% for sensitivity and from 67% to 100% for specificity. The discriminative performance in terms of AUPRC ranged from 0·57 to 1·00 in the five automated deep learning models. In an external validation using the Edinburgh Dermofit Library dataset, the automated deep learning model showed an AUPRC of 0·47, with a sensitivity of 49% and a positive predictive value of 52%. Interpretation All models, except the automated deep learning model trained on the multilabel classification task of the NIH CXR14 dataset, showed comparable discriminative performance and diagnostic properties to state-of-the-art performing deep learning algorithms. The performance in the external validation study was low. The quality of the open-access datasets (including insufficient information about patient flow and demographics) and the absence of measurement for precision, such as confidence intervals, constituted the major limitations of this study. The availability of automated deep learning platforms provide an opportunity for the medical community to enhance their understanding in model development and evaluation. Although the derivation of classification models without requiring a deep understanding of the mathematical, statistical, and programming principles is attractive, comparable performance to expertly designed models is limited to more elementary classification tasks. Furthermore, care should be placed in adhering to ethical principles when using these automated models to avoid discrimination and causing harm. Future studies should compare several application programming interfaces on thoroughly curated datasets. Funding National Institute for Health Research and Moorfields Eye Charity.

Purpose\ud To determine the time and risk factors for developing proliferative diabetic retinopathy (PDR) and vitreous hemorrhage (VH).\ud \ud Design\ud Multicenter, national cohort study.\ud \ud Methods\ud Anonymized data of 50 254 patient eyes with diabetes mellitus at 19 UK hospital eye services were extracted at the initial and follow-up visits between 2007 and 2014. Time to progression of PDR and VH were calculated with Cox regression after stratifying by baseline diabetic retinopathy (DR) severity and adjusting for age, sex, race, and starting visual acuity.\ud \ud Results\ud Progression to PDR in 5 years differed by baseline DR: no DR (2.2%), mild (13.0%), moderate (27.2%), severe nonproliferative diabetic retinopathy (NPDR) (45.5%). Similarly, 5-year progression to VH varied by baseline DR: no DR (1.1%), mild (2.9%), moderate (7.3%), severe NPDR (9.8%). Compared with no DR, the patient eyes that presented with mild, moderate, and severe NPDR were 6.71, 14.80, and 28.19 times more likely to develop PDR, respectively. In comparison to no DR, the eyes with mild, moderate, and severe NPDR were 2.56, 5.60, and 7.29 times more likely to develop VH, respectively. In severe NPDR, the eyes with intraretinal microvascular abnormalities (IRMA) had a significantly increased hazard ratio (HR) of developing PDR (HR 1.77, 95% confidence interval [CI] 1.25–2.49, P = .0013) compared with those with venous beading, whereas those with 4-quadrant dot-blot hemorrhages (4Q DBH) had 3.84 higher HR of developing VH (95% CI 1.39–10.62, P = .0095).\ud \ud Conclusions\ud Baseline severities and features of initial DR are prognostic for PDR development. IRMA increases risk of PDR whereas 4Q DBH increases risk of VH.

Punctate inner choroidopathy (PIC), an idiopathic inflammatory multifocal chorioretinopathy that predominantly affects young myopic women, appears to be relatively rare, but there are limited data to support accurate estimates of prevalence, and it is likely that the condition is underdiagnosed. The etiological relationship between PIC and other conditions within the "white dot syndromes" group remains uncertain. We, like others, would suggest that PIC and multifocal choroiditis with panuveitis represent a single disease process that is modified by host factors (including host immunoregulation) to cause the range of clinical phenotypes seen. The impact of PIC on the patient is highly variable, with outcome ranging from complete spontaneous recovery to bilateral severe sight loss. Detection and monitoring have been greatly facilitated by modern scanning techniques, especially optical coherence tomography and autofluorescence imaging and may be enhanced by coregistration of sequential images to detect change over time. Depending on the course of disease and nature of complications, appropriate treatment may range from observation to systemic immunosuppression and antiangiogenic therapies. PIC is a challenging condition where treatment has to be tailored to the patient's individual circumstances, the extent of disease, and the risk of progression.

Background: Poor access to existing care for diabetic retinopathy (DR) limits effectiveness of proven treatments. We examined whether outreach screening in rural China improves equity of access. Methods: We compared prevalence of female sex, age >=65 years, primary education or below, and requiring referral care for DR between three cohorts with diabetes examined for DR in neighboring areas of Guangdong, China: passive case detection at secondary-level hospitals (n=193); persons screened during primary-level DR outreach (n=185); and individuals with newly- or previously-diagnosed diabetes in a population survey (n=579). The latter reflected the “ideal” reach of a screening program. Findings: Compared to the population cohort, passive case detection reached fewer women (50·8% vs. 62·3%, p = 0·006), older adults (37·8% vs. 51·3%, p 0.300) and persons aged >= 65 years (49.5% vs 51.3%, p=0.723) in the outreach screening and population cohorts did not differ significantly. Prevalence of requiring referral care for DR was significantly higher in the outreach screening cohort (28·0%) than the population (14·0%) and passive case detection cohorts (7·3%, p

Background: Studies during the COVID-19 pandemic have suggested patient characteristics with increased risk of death. Most studies have not included populations which reflect an urban UK demographic but emerging case reports have suggested poorer outcomes in certain ethnic groups. It was hypothesised that people from South Asian ethnic groups would be more susceptible to severe manifestations of COVID-19. Methods: Patients with confirmed SARS-CoV-2 infection by positive polymerase chain reaction testing and requiring admission to University Hospitals Birmingham NHS Foundation Trust in Birmingham UK between 10th March 2020 and 17th April 2020 were included. Demographics, ethnicity, baseline co-morbidities, social deprivation index and outcome (death within the censor date) were assessed and Cox regression analysis conducted. Using observed sex-specific age distributions of COVID-19 admissions/deaths and 2011 census data for Birmingham/Solihull, expected numbers of admissions and deaths were estimated and ratios of observed to expected numbers calculated, providing standardised admission ratios (SAR) and standardised mortality ratios (SMR). Results: 2217 patients admitted to UHB with a proven diagnosis of COVID-19 were included. 58.2% were male, 69.5% White and the majority (80.2%) had co-morbidities. 18.5% were of South Asian ethnicity, and these patients were more likely to be younger (median age 61 vs.77), have no co-morbidities (27.8% vs. 16.6%) but a higher prevalence of diabetes mellitus (48.1% vs 28.2%) than White patients. SAR and SMR suggested more admissions and deaths in South Asian patients than would be predicted. These patients were more likely to present with severe disease. South Asian ethnicity was associated with an increased risk of death (Hazard Ratio 1.67 (95%CI 1.34 – 2.10)) after adjusting for age, sex, deprivation and comorbidities. Interpretation: Current evidence suggests those of South Asian ethnicity may be at risk of worse COVID019 outcomes, further studies needs to establish the underlying mechanistic pathways. Funding Statement: HDRUK Hub PIONEER Declaration of Interests: S Gallier, C Mainey, P Nightingale, D McNulty, A Kolesnyk, M Ahmed, H Crothers, F Evison, A Liaqat, L Irshad, M. Harris, T Nabavi, P Cockwell, D Pagano, report no conflicts of interest. S Ballreports funding support from the HDFR-UK, K Reeves reports funding support from the NIHR, E Sapey reports funding support from the MRC, Wellcome Trust, NIHR and British Lung Foundation. K. Nirantharakumar reports funding from MRC, Wellcome Trust, NIHR, Vifor and AstraZeneca. A.K Dennistonreports funding from HDR-UK, Wellcome Trust and Fight for Sight. Ethics Approval Statement: This retrospective cohort study, using prospectively collected data was conducted as part of DECOVID, an HRA and London - City & East Research Ethics Committee approved research database (Ethics number 20/HRA/1689).

Purpose To investigate if previous intravitreal therapy is a predictor of posterior capsule rupture (PCR) during cataract surgery. Design Multicenter, national electronic medical record (EMR) database study with univariate and multivariate regression modeling. Participants A total of 65 836 eyes of 44 635 patients undergoing cataract surgery. Methods Anonymized data were extracted for eyes undergoing cataract surgery from 20 hospitals using the same EMR for cases performed between 2004 and 2014. Variables included as possible risk indicators for PCR were age, sex, number of previous intravitreal injections, indication for intravitreal therapy, grade of healthcare professional administering intravitreal therapy, advanced cataract, and cataract surgeon grade. Main Outcome Measures Presence or absence of posterior capsular rupture during cataract surgery. Results Data were available on 65 836 cataract operations, of which 1935 had undergone previous intravitreal therapy (2.9%). In univariate regression analyses, patient age, advanced cataract, junior cataract surgeon grade, and number of previous intravitreal injections were significant predictors of PCR. By considering the number of previous intravitreal injections as a continuous variable, the odds ratio for PCR per intravitreal injection was 1.04 ( P = 0.016) after adjusting for age, advanced cataract, and cataract surgeon grade. Repeat analysis considering intravitreal injections as a categoric variable showed 10 or more previous injections were associated with a 2.59 times higher likelihood of PCR ( P = 0.003) after again adjusting for other significant independent predictors. Conclusions Previous intravitreal therapy is associated with a higher likelihood of PCR during cataract surgery. This study provides data to help inform surgeons and patients about the risk of complications when undergoing cataract surgery after multiple prior intravitreal injections. Further investigation is required to determine the cause behind the increased PCR risk.

Background: Deep learning offers considerable promise for medical diagnostics. In this review, we evaluated the diagnostic accuracy of deep learning (DL) algorithms versus health care professionals (HCPs) in classifying diseases from medical imaging. Methods: We searched (Pre-)Medline, Embase, Science Citation Index, Conference Proceedings Citation Index, and arXiv from 01 January 2012 until 31 May 2018. Studies comparing the diagnostic performance of DL models and HCPs, for any pre-specified condition based on medical imaging material, were included. We extracted binary diagnostic accuracy data and constructed contingency tables at the reported thresholds to derive the outcomes of interest: sensitivity and specificity. Studies undertaking an out-of-sample validation were included in a meta-analysis. Results: 24 studies, from a starting number of 19889, compared DL models with HCPs. 22 studies provided enough data to construct contingency tables, enabling calculation of test accuracy. The mean sensitivity for DL models was 78% (range 13 - 100%), and mean specificity was 86% (range 51 - 100%). An out-of-sample external validation was performed by 5 studies and were therefore included in the meta-analysis. We found a pooled sensitivity of 86% (95% CI: 84 - 88%) for DL models and 93% (95% CI: 87 - 97%) for HCPs, and a pooled specificity of 88% (95% CI: 84 - 92%) for DL models and 87% (95% CI: 84 - 89%) for HCPs. Conclusion: Our review found the diagnostic performance of deep learning models to be similar to health care professionals. A major finding was the poor reporting and potential biases arising from study design that limited reliable interpretation of the reported diagnostic accuracy. New reporting standards which address specific challenges of deep learning could improve future studies, enabling greater confidence in the results of future evaluations of this promising technology. Funding Statement: The authors state: "None" Declaration of Interests: All authors have completed the ICMJE uniform disclosure form online (available on request from the corresponding author) and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work. Ethics Approval Statement: The authors state: "Not required." The authors utilized PRISMA and MOOSE protocols.

Deep learning has huge potential to transform healthcare. However, significant expertise is required to train such models and this is a significant blocker for their translation into clinical practice. In this study, we therefore sought to evaluate the use of automated deep learning software to develop medical image diagnostic classifiers by healthcare professionals with limited coding – and no deep learning – expertise.We used five publicly available open-source datasets: (i) retinal fundus images (MESSIDOR); (ii) optical coherence tomography (OCT) images (Guangzhou Medical University/Shiley Eye Institute, Version 3); (iii) images of skin lesions (Human against Machine (HAM)10000) and (iv) both paediatric and adult chest X-ray (CXR) images (Guangzhou Medical University/Shiley Eye Institute, Version 3 and the National Institute of Health (NIH)14 dataset respectively) to separately feed into a neural architecture search framework that automatically developed a deep learning architecture to classify common diseases. Sensitivity (recall), specificity and positive predictive value (precision) were used to evaluate the diagnostic properties of the models. The discriminative performance was assessed using the area under the precision recall curve (AUPRC). In the case of the deep learning model developed on a subset of the HAM10000 dataset, we performed external validation using the Edinburgh Dermofit Library dataset.Diagnostic properties and discriminative performance from internal validations were high in the binary classification tasks (range: sensitivity of 73.3-97.0%, specificity of 67-100% and AUPRC of 0.87-1). In the multiple classification tasks, the diagnostic properties ranged from 38-100% for sensitivity and 67-100% for specificity. The discriminative performance in terms of AUPRC ranged from 0.57 to 1 in the five automated deep learning models. In an external validation using the Edinburgh Dermofit Library dataset, the automated deep learning model showed an AUPRC of 0.47, with a sensitivity of 49% and a positive predictive value of 52%. The quality of the open-access datasets used in this study (including the lack of information about patient flow and demographics) and the absence of measurement for precision, such as confidence intervals, constituted the major limitation of this study.All models, except for the automated deep learning model trained on the multi-label classification task of the NIH CXR14 dataset, showed comparable discriminative performance and diagnostic properties to state-of-the-art performing deep learning algorithms. The performance in the external validation study was low. The availability of automated deep learning may become a cornerstone for the democratization of sophisticated algorithmic modelling in healthcare as it allows the derivation of classification models without requiring a deep understanding of the mathematical, statistical and programming principles. Future studies should compare several application programming interfaces on thoroughly curated datasets.

Systemic lupus erythematosus (SLE) is a life-threatening multisystem inflammatory condition that may affect almost any part of the eye. We provide an update for the practicing ophthalmologist comprising a systematic review of the recent literature presented in the context of current knowledge of the pathogenesis, diagnosis, and treatment of this condition. We review recent advances in the understanding of the influence of genetic and environmental factors on the development of SLE. Recent changes in the diagnostic criteria for SLE are considered. We assess the potential for novel molecular biomarkers to find a clinical application in disease diagnosis and stratification and in the development of therapeutic agents. We discuss limited forms of SLE and their differentiation from other collagen vascular disorders and review recent evidence underlying the use of established and novel therapeutics in this condition, including specific implications regarding monitoring for ocular toxicity associated with antimalarials.

Purpose To obtain measurements of vitreous signal intensity from optical coherence tomography (OCT) image sets in patients with uveitis, with the aim of developing an objective, quantitative marker of inflammatory activity in patients with this disease. Design Retrospective, observational case-control series. Participants Thirty patients (30 eyes) with vitreous haze secondary to intermediate, posterior, or panuveitis; 12 patients (12 eyes) with uveitis but without evidence of vitreous haze; and 18 patients (18 eyes) without intraocular inflammation or vitreoretinal disease. Methods Clinical and demographic characteristics were recorded, including visual acuity (VA), diagnosis, and anatomic type of uveitis. In each eye, the anterior chamber (AC) was graded for cellular activity and flare according to standardized protocols. The presence and severity of vitreous haze were classified according to the National Eye Institute system. Spectral-domain OCT images were analyzed using custom software. This software provided an "absolute" measurement of vitreous signal intensity, which was then compared with that of the retinal pigment epithelium (RPE), generating an optical density ratio with arbitrary units ("VIT/RPE-Relative Intensity"). Main Outcome Measures Correlation between clinical vitreous haze scores and OCT-derived measurements of vitreous signal intensity. Results The VIT/RPE-Relative Intensity was significantly higher in uveitic eyes with known vitreous haze (0.150) than in uveitic eyes without haze or in healthy controls (0.0767, P = 0.0001). The VIT/RPE-Relative Intensity showed a significant, positive correlation with clinical vitreous haze scores ( r = 0.566, P = 0.0001). Other ocular characteristics significantly associated with VIT/RPE-Relative Intensity included VA ( r = 0.573, P = 0.0001), AC cells ( r = 0.613, P = 0.0001), and AC flare ( r = 0.385, P = 0.003). Measurement of VIT/RPE-Relative Intensity showed a good degree of intergrader reproducibility (95% limits of agreement, −0.019 to 0.016). Conclusions These results provide preliminary evidence that OCT-derived measurements of vitreous signal intensity may be useful as an outcome measure in patients with uveitis. If validated in future studies, such measures may serve as an objective, quantitative disease activity end point, with the potential to improve the "signal:noise" ratio of clinical trials in this area, thus enabling smaller studies for the same power. The incorporation of automated vitreous analysis in commercial OCT systems may, in turn, facilitate monitoring and re-treatment of patients with uveitis in clinical practice.