Search

Objective: To determine the appropriateness of ophthalmology recommendations from an online chat-based artificial intelligence model to ophthalmology questions. Patients and Methods: Cross-sectional qualitative study from April 1, 2023, to April 30, 2023. A total of 192 questions were generated spanning all ophthalmic subspecialties. Each question was posed to a large language model (LLM) 3 times. The responses were graded by appropriate subspecialists as appropriate, inappropriate, or unreliable in 2 grading contexts. The first grading context was if the information was presented on a patient information site. The second was an LLM-generated draft response to patient queries sent by the electronic medical record (EMR). Appropriate was defined as accurate and specific enough to serve as a surrogate for physician-approved information. Main outcome measure was percentage of appropriate responses per subspecialty. Results: For patient information site-related questions, the LLM provided an overall average of 79% appropriate responses. Variable rates of average appropriateness were observed across ophthalmic subspecialties for patient information site information ranging from 56% to 100%: cataract or refractive (92%), cornea (56%), glaucoma (72%), neuro-ophthalmology (67%), oculoplastic or orbital surgery (80%), ocular oncology (100%), pediatrics (89%), vitreoretinal diseases (86%), and uveitis (65%). For draft responses to patient questions via EMR, the LLM provided an overall average of 74% appropriate responses and varied by subspecialty: cataract or refractive (85%), cornea (54%), glaucoma (77%), neuro-ophthalmology (63%), oculoplastic or orbital surgery (62%), ocular oncology (90%), pediatrics (94%), vitreoretinal diseases (88%), and uveitis (55%). Stratifying grades across health information categories (disease and condition, risk and prevention, surgery-related, and treatment and management) showed notable but insignificant variations, with disease and condition often rated highest (72% and 69%) for appropriateness and surgery-related (55% and 51%) lowest, in both contexts. Conclusion: This LLM reported mostly appropriate responses across multiple ophthalmology subspecialties in the context of both patient information sites and EMR-related responses to patient questions. Current LLM offerings require optimization and improvement before widespread clinical use.

Purpose: To develop and validate machine learning (ML) models to predict choroidal nevus transformation to melanoma based on multimodal imaging at initial presentation. Design: Retrospective multicenter study. Participants: Patients diagnosed with choroidal nevus on the Ocular Oncology Service at Wills Eye Hospital (2007–2017) or Mayo Clinic Rochester (2015–2023). Methods: Multimodal imaging was obtained, including fundus photography, fundus autofluorescence, spectral domain OCT, and B-scan ultrasonography. Machine learning models were created (XGBoost, LGBM, Random Forest, Extra Tree) and optimized for area under receiver operating characteristic curve (AUROC). The Wills Eye Hospital cohort was used for training and testing (80% training–20% testing) with fivefold cross validation. The Mayo Clinic cohort provided external validation. Model performance was characterized by AUROC and area under precision–recall curve (AUPRC). Models were interrogated using SHapley Additive exPlanations (SHAP) to identify the features most predictive of conversion from nevus to melanoma. Differences in AUROC and AUPRC between models were tested using 10 000 bootstrap samples with replacement and results. Main Outcome Measures: Area under receiver operating curve and AUPRC for each ML model. Results: There were 2870 nevi included in the study, with conversion to melanoma confirmed in 128 cases. Simple AI Nevus Transformation System (SAINTS; XGBoost) was the top-performing model in the test cohort [pooled AUROC 0.864 (95% confidence interval (CI): 0.864–0.865), pooled AUPRC 0.244 (95% CI: 0.243–0.246)] and in the external validation cohort [pooled AUROC 0.931 (95% CI: 0.930–0.931), pooled AUPRC 0.533 (95% CI: 0.531–0.535)]. Other models also had good discriminative performance: LGBM (test set pooled AUROC 0.831, validation set pooled AUROC 0.815), Random Forest (test set pooled AUROC 0.812, validation set pooled AUROC 0.866), and Extra Tree (test set pooled AUROC 0.826, validation set pooled AUROC 0.915). A model including only nevi with at least 5 years of follow-up demonstrated the best performance in AUPRC (test: pooled 0.592 (95% CI: 0.590–0.594); validation: pooled 0.656 [95% CI: 0.655–0.657]). The top 5 features in SAINTS by SHAP values were: tumor thickness, largest tumor basal diameter, tumor shape, distance to optic nerve, and subretinal fluid extent. Conclusions: We demonstrate accuracy and generalizability of a ML model for predicting choroidal nevus transformation to melanoma based on multimodal imaging. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Purpose: To determine the effectiveness of aerosol-delivered methotrexate (AD-MTx) in a large-animal (porcine) model of proliferative vitreoretinopathy (PVR). Design: Prospective, randomized, interventional, double-masked, controlled, large-animal study with predetermined clinical and histopathologic outcome criteria. Controls: Half of the pigs were randomly assigned to receive an identical volume of aerosol-delivered normal saline (AD-NS) using identical delivery systems and treatment intervals. Methods: Proliferative vitreoretinopathy was surgically induced in 16 pigs (8 males and 8 females), randomly assigned to receive 2 doses (group A) or 3 doses (group B) of either AD-MTx (1.6 mg/0.4 ml) or normal saline (AD-NS). Group A pigs were euthanized at week 2 (n = 8), and group B pigs were euthanized at week 3 (n = 8). Masked clinical PVR scores (0–6) by a vitreoretinal surgeon and histopathology PVR scores (0–8) by a masked ophthalmic pathologist were used to determine outcomes. Main Outcome Measures: The mean, combined clinical and histopathology scores (both anterior and posterior) were used to determine the overall treatment effect between the groups. Results: The mean masked score (± standard deviation) when all grading end points (clinical + histopathology) were combined was a mean of 8.0 ± 2.3 in the AD-MTx group versus a higher 9.9 ± 2.0 in the AD-NS control group (P = 0.05). The clinical score was 3.88 ± 1.2 in the AD-MTx group versus 4.63 ± 1.6 in the AD-NS group (P = 0.16). The histopathology score for anterior PVR was 2.5 ± 0.8 in the AD-MTx group versus 2.5 ± 0.5 in the AD-NS group (P = 0.50), and the posterior PVR was 1.63 ± 1.6 in the AD-MTx group versus 2.75 ± 1.3 in the AD-NS group (P = 0.07). When the frequency of methotrexate dosing in group A (2 doses) was compared with that in group B (3 doses), the mean score was 8.75 versus 9.13 (P = 0.38), respectively, suggesting an insignificant difference. Conclusions: After surgical induction of PVR in an aggressive, high-risk, large-animal model, AD-MTx reduced posterior PVR formation compared with AD-NS. Additional dosing at week 3 did not improve the outcomes. No difference in anterior PVR formation was noted with intervention. This novel drug delivery system has implications for PVR reduction and warrants further investigation. Financial Disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.