Search

Social determinants of health (SDoH) play a critical role in patient outcomes, yet their documentation is often missing or incomplete in the structured data of electronic health records (EHRs). Large language models (LLMs) could enable high-throughput extraction of SDoH from the EHR to support research and clinical care. However, class imbalance and data limitations present challenges for this sparsely documented yet critical information. Here, we investigated the optimal methods for using LLMs to extract six SDoH categories from narrative text in the EHR: employment, housing, transportation, parental status, relationship, and social support. The best-performing models were fine-tuned Flan-T5 XL for any SDoH mentions (macro-F1 0.71), and Flan-T5 XXL for adverse SDoH mentions (macro-F1 0.70). Adding LLM-generated synthetic data to training varied across models and architecture, but improved the performance of smaller Flan-T5 models (delta F1 + 0.12 to +0.23). Our best-fine-tuned models outperformed zero- and few-shot performance of ChatGPT-family models in the zero- and few-shot setting, except GPT4 with 10-shot prompting for adverse SDoH. Fine-tuned models were less likely than ChatGPT to change their prediction when race/ethnicity and gender descriptors were added to the text, suggesting less algorithmic bias (p < 0.05). Our models identified 93.8% of patients with adverse SDoH, while ICD-10 codes captured 2.0%. These results demonstrate the potential of LLMs in improving real-world evidence on SDoH and assisting in identifying patients who could benefit from resource support. [ABSTRACT FROM AUTHOR]

Purpose To develop, externally test, and evaluate clinical acceptability of a deep learning pediatric brain tumor segmentation model using stepwise transfer learning. Materials and Methods In this retrospective study, the authors leveraged two T2-weighted MRI datasets (May 2001 through December 2015) from a national brain tumor consortium ( n = 184; median age, 7 years [range, 1-23 years]; 94 male patients) and a pediatric cancer center ( n = 100; median age, 8 years [range, 1-19 years]; 47 male patients) to develop and evaluate deep learning neural networks for pediatric low-grade glioma segmentation using a stepwise transfer learning approach to maximize performance in a limited data scenario. The best model was externally tested on an independent test set and subjected to randomized blinded evaluation by three clinicians, wherein they assessed clinical acceptability of expert- and artificial intelligence (AI)-generated segmentations via 10-point Likert scales and Turing tests. Results The best AI model used in-domain stepwise transfer learning (median Dice score coefficient, 0.88 [IQR, 0.72-0.91] vs 0.812 [IQR, 0.56-0.89] for baseline model; P = .049). With external testing, the AI model yielded excellent accuracy using reference standards from three clinical experts (median Dice similarity coefficients: expert 1, 0.83 [IQR, 0.75-0.90]; expert 2, 0.81 [IQR, 0.70-0.89]; expert 3, 0.81 [IQR, 0.68-0.88]; mean accuracy, 0.82). For clinical benchmarking ( n = 100 scans), experts rated AI-based segmentations higher on average compared with other experts (median Likert score, 9 [IQR, 7-9] vs 7 [IQR 7-9]) and rated more AI segmentations as clinically acceptable (80.2% vs 65.4%). Experts correctly predicted the origin of AI segmentations in an average of 26.0% of cases. Conclusion Stepwise transfer learning enabled expert-level automated pediatric brain tumor autosegmentation and volumetric measurement with a high level of clinical acceptability. Keywords: Stepwise Transfer Learning, Pediatric Brain Tumors, MRI Segmentation, Deep Learning Supplemental material is available for this article . © RSNA, 2024.

Purpose: Programmed cell death protein 1 (PD-1) expression on CD8+TIM-3-LAG-3- tumor-infiltrating cells predicts positive response to PD-1 blockade in metastatic clear-cell renal cell carcinoma (mccRCC). Because inhibition of PD-1 signaling in regulatory T cells (Treg) augments their immunosuppressive function, we hypothesized that PD-1 expression on tumor-infiltrating Tregs would predict resistance to PD-1 inhibitors., Experimental Design: PD-1+ Tregs were phenotyped using multiparametric immunofluorescence in ccRCC tissues from the CheckMate-025 trial (nivolumab: n = 91; everolimus: n = 90). Expression of CD8, PD-1, TIM-3, and LAG-3 was previously determined (Ficial and colleagues, 2021). Clinical endpoints included progression-free survival (PFS), overall survival (OS), and objective response rate (ORR)., Results: In the nivolumab (but not everolimus) arm, high percentage of PD-1+ Tregs was associated with shorter PFS (3.19 vs. 5.78 months; P = 0.021), shorter OS (18.1 vs. 27.7 months; P = 0.013) and marginally lower ORR (12.5% vs. 31.3%; P = 0.059). An integrated biomarker (PD-1 Treg/CD8 ratio) was developed by calculating the ratio between percentage of PD-1+Tregs (marker of resistance) and percentage of CD8+PD-1+TIM-3-LAG-3- cells (marker of response). In the nivolumab (but not everolimus) arm, patients with high PD-1 Treg/CD8 ratio experienced shorter PFS (3.48 vs. 9.23 months; P < 0.001), shorter OS (18.14 vs. 38.21 months; P < 0.001), and lower ORR (15.69% vs. 40.00%; P = 0.009). Compared with the individual biomarkers, the PD-1 Treg/CD8 ratio showed improved ability to predict outcomes to nivolumab versus everolimus., Conclusions: PD-1 expression on Tregs is associated with resistance to PD-1 blockade in mccRCC, suggesting that targeting Tregs may synergize with PD-1 inhibition. A model that integrates PD-1 expression on Tregs and CD8+TIM-3-LAG-3- cells has higher predictive value., (©2023 American Association for Cancer Research.)