Search

Despite the predominance of the evidence-based medicine paradigm, a fundamental incongruity remains: Evidence is derived from groups of people, yet medical decisions are made by and for individuals. Randomization ensures the comparability of treatment groups within a clinical trial, which allows for unbiased estimation of average treatment effects. If we treated groups of patients instead of individuals, or if patients with the same disease were identical to one another in all factors that determined the harms and the benefits of therapy, then these group-level averages would make a perfectly sound foundation for medical decision-making. But patients differ from one another in many ways that determine the likelihood of an outcome, both with and without a treatment. Nevertheless, popular approaches to evidence-based medicine have encouraged a reliance on the average treatment effects estimated from clinical trials and meta-analysis as guides to decision-making for individuals. Here, we discuss the limitations of this approach as well as limitations of conventional, one-variable-at-a-time subgroup analysis; finally, we discuss the rationale for "predictive" approaches to heterogeneous treatment effects. Predictive approaches to heterogeneous treatment effects combine methods for causal inference (e.g. randomization) with methods for prediction that permit inferences about which patients are likely to benefit and which are not, taking into account multiple relevant variables simultaneously to yield "personalized" estimates of benefit-harm trade-offs. We focus on risk modeling approaches, which rely on the mathematical dependence of the absolute treatment effect with the baseline risk, which varies substantially "across patients" in most trials. While there are a number of examples of risk modeling approaches that have been practice-changing, risk modeling does not provide ideal estimates of individual treatment effects, since risk modeling does not account for how individual variables might modify the effects of therapy. In "effect modeling," prediction models are developed directly on clinical trial data, including terms for treatment and treatment effect interactions. These more flexible approaches may better uncover individualized treatment effects, but are also prone to overfitting when dimensionality is high, power is low, and there is limited prior knowledge about effect modifiers.

Background: Measuring the performance of models that predict individualized treatment effect is challenging because the outcomes of two alternative treatments are inherently unobservable in one patient. The C-for-benefit was proposed to measure discriminative ability. However, measures of calibration and overall performance are still lacking. We aimed to propose metrics of calibration and overall performance for models predicting treatment effect in randomized clinical trials (RCTs)., Methods: Similar to the previously proposed C-for-benefit, we defined observed pairwise treatment effect as the difference between outcomes in pairs of matched patients with different treatment assignment. We match each untreated patient with the nearest treated patient based on the Mahalanobis distance between patient characteristics. Then, we define the E avg -for-benefit, E 50 -for-benefit, and E 90 -for-benefit as the average, median, and 90 th quantile of the absolute distance between the predicted pairwise treatment effects and local-regression-smoothed observed pairwise treatment effects. Furthermore, we define the cross-entropy-for-benefit and Brier-for-benefit as the logarithmic and average squared distance between predicted and observed pairwise treatment effects. In a simulation study, the metric values of deliberately "perturbed models" were compared to those of the data-generating model, i.e., "optimal model". To illustrate these performance metrics, different modeling approaches for predicting treatment effect are applied to the data of the Diabetes Prevention Program: 1) a risk modelling approach with restricted cubic splines; 2) an effect modelling approach including penalized treatment interactions; and 3) the causal forest., Results: As desired, performance metric values of "perturbed models" were consistently worse than those of the "optimal model" (E avg -for-benefit ≥ 0.043 versus 0.002, E 50 -for-benefit ≥ 0.032 versus 0.001, E 90 -for-benefit ≥ 0.084 versus 0.004, cross-entropy-for-benefit ≥ 0.765 versus 0.750, Brier-for-benefit ≥ 0.220 versus 0.218). Calibration, discriminative ability, and overall performance of three different models were similar in the case study. The proposed metrics were implemented in a publicly available R-package "HTEPredictionMetrics"., Conclusion: The proposed metrics are useful to assess the calibration and overall performance of models predicting treatment effect in RCTs., (© 2023. The Author(s).)

Objectives: To (1) explore trends of risk of bias (ROB) in prediction research over time following key methodological publications, using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and (2) assess the inter-rater agreement of the PROBAST., Study Design and Setting: PubMed and Web of Science were searched for reviews with extractable PROBAST scores on domain and signaling question (SQ) level. ROB trends were visually correlated with yearly citations of key publications. Inter-rater agreement was assessed using Cohen's Kappa., Results: One hundred and thirty nine systematic reviews were included, of which 85 reviews (containing 2,477 single studies) on domain level and 54 reviews (containing 2,458 single studies) on SQ level. High ROB was prevalent, especially in the Analysis domain, and overall trends of ROB remained relatively stable over time. The inter-rater agreement was low, both on domain (Kappa 0.04-0.26) and SQ level (Kappa -0.14 to 0.49)., Conclusion: Prediction model studies are at high ROB and time trends in ROB as assessed with the PROBAST remain relatively stable. These results might be explained by key publications having no influence on ROB or recency of key publications. Moreover, the trend may suffer from the low inter-rater agreement and ceiling effect of the PROBAST. The inter-rater agreement could potentially be improved by altering the PROBAST or providing training on how to apply the PROBAST., Competing Interests: Declaration of competing interest The work on this study by R.J.J. and M.v.D. was supported by a grant from the Dutch Kidney Foundation (20OK016). All other authors declare no conflict of interest., (Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.)

Objective: To estimate the risk of a patient with osteoarthritis (OA) developing chronic opioid use (COU) within 1 year of a new opioid prescription by using electronic health record (EHR) data and predictive models., Methods: We used EHR data from 13 health care organizations to identify patients with OA with an opioid prescription between March 1, 2017 and February 28, 2019 and no record of opioid use in the prior 6 months. We evaluated 4 machine learning models to estimate patients' risk of COU (≥3 prescriptions ≥84 days, maximum gap ≤60 days). We also estimated the transportability of models to organizations outside the training set., Results: The cohort consisted of 33,894 patients with OA, of whom 2,925 (8.6%) developed COU within 1 year. All models demonstrated good discrimination, with the best-performing model (random forest) achieving an area under the receiver operating characteristic curve (AUC) of 0.728 (95% CI 0.711-0.745), but the simplest regression model performed nearly as well (AUC 0.717 [95% CI 0.699-0.734]). Predicted risk deciles spanned a range of 2% risk for the 10th percentile to 18% risk for the 90th percentile for well-calibrated models. Models showed highly variable discrimination across organizations (AUC 0.571-0.842)., Conclusions: We found that EHR-based predictive models could estimate the risk of future COU among patients with OA to help inform care decisions. Black-box methods did not have significant advantages over more interpretable models. Care should be taken when extending all models into organizations not included in model training because of a high variability in performance across held-out organizations., (© 2022 The Authors. Arthritis Care & Research published by Wiley Periodicals LLC on behalf of American College of Rheumatology.)

Background: People with comorbidities are underrepresented in clinical trials. Empirical estimates of treatment effect modification by comorbidity are lacking, leading to uncertainty in treatment recommendations. We aimed to produce estimates of treatment effect modification by comorbidity using individual participant data (IPD)., Methods and Findings: We obtained IPD for 120 industry-sponsored phase 3/4 trials across 22 index conditions (n = 128,331). Trials had to be registered between 1990 and 2017 and have recruited ≥300 people. Included trials were multicentre and international. For each index condition, we analysed the outcome most frequently reported in the included trials. We performed a two-stage IPD meta-analysis to estimate modification of treatment effect by comorbidity. First, for each trial, we modelled the interaction between comorbidity and treatment arm adjusted for age and sex. Second, for each treatment within each index condition, we meta-analysed the comorbidity-treatment interaction terms from each trial. We estimated the effect of comorbidity measured in 3 ways: (i) the number of comorbidities (in addition to the index condition); (ii) presence or absence of the 6 commonest comorbid diseases for each index condition; and (iii) using continuous markers of underlying conditions (e.g., estimated glomerular filtration rate (eGFR)). Treatment effects were modelled on the usual scale for the type of outcome (absolute scale for numerical outcomes, relative scale for binary outcomes). Mean age in the trials ranged from 37.1 (allergic rhinitis trials) to 73.0 (dementia trials) and percentage of male participants range from 4.4% (osteoporosis trials) to 100% (benign prostatic hypertrophy trials). The percentage of participants with 3 or more comorbidities ranged from 2.3% (allergic rhinitis trials) to 57% (systemic lupus erythematosus trials). We found no evidence of modification of treatment efficacy by comorbidity, for any of the 3 measures of comorbidity. This was the case for 20 conditions for which the outcome variable was continuous (e.g., change in glycosylated haemoglobin in diabetes) and for 3 conditions in which the outcomes were discrete events (e.g., number of headaches in migraine). Although all were null, estimates of treatment effect modification were more precise in some cases (e.g., sodium-glucose co-transporter-2 (SGLT2) inhibitors for type 2 diabetes-interaction term for comorbidity count 0.004, 95% CI -0.01 to 0.02) while for others credible intervals were wide (e.g., corticosteroids for asthma-interaction term -0.22, 95% CI -1.07 to 0.54). The main limitation is that these trials were not designed or powered to assess variation in treatment effect by comorbidity, and relatively few trial participants had >3 comorbidities., Conclusions: Assessments of treatment effect modification rarely consider comorbidity. Our findings demonstrate that for trials included in this analysis, there was no empirical evidence of treatment effect modification by comorbidity. The standard assumption used in evidence syntheses is that efficacy is constant across subgroups, although this is often criticised. Our findings suggest that for modest levels of comorbidities, this assumption is reasonable. Thus, trial efficacy findings can be combined with data on natural history and competing risks to assess the likely overall benefit of treatments in the context of comorbidity., Competing Interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: SD received fees from the Association of the British Pharmaceutical Industry (ABPI) for delivery of a Masterclass (unrelated to this work). The other authors have declared that no competing interests exist., (Copyright: © 2023 Hanlon et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Introduction: Clinical prediction models (CPMs) for coronavirus disease 2019 (COVID-19) may support clinical decision making, treatment, and communication. However, attitudes about using CPMs for COVID-19 decision making are unknown., Methods: Online focus groups and interviews were conducted among health care providers, survivors of COVID-19, and surrogates (i.e., loved ones/surrogate decision makers) in the United States and the Netherlands. Semistructured questions explored experiences about clinical decision making in COVID-19 care and facilitators and barriers for implementing CPMs., Results: In the United States, we conducted 4 online focus groups with 1) providers and 2) surrogates and survivors of COVID-19 between January 2021 and July 2021. In the Netherlands, we conducted 3 focus groups and 4 individual interviews with 1) providers and 2) surrogates and survivors of COVID-19 between May 2021 and July 2021. Providers expressed concern about CPM validity and the belief that patients may interpret CPM predictions as absolute. They described CPMs as potentially useful for resource allocation, triaging, education, and research. Several surrogates and people who had COVID-19 were not given prognostic estimates but believed this information would have supported and influenced their decision making. A limited number of participants felt the data would not have applied to them and that they or their loved ones may not have survived, as poor prognosis may have suggested withdrawal of treatment., Conclusions: Many providers had reservations about using CPMs for people with COVID-19 due to concerns about CPM validity and patient-level interpretation of the outcome predictions. However, several people who survived COVID-19 and their surrogates indicated that they would have found this information useful for decision making. Therefore, information provision may be needed to improve provider-level comfort and patient and surrogate understanding of CPMs., Highlights: While clinical prediction models (CPMs) may provide an objective means of assessing COVID-19 prognosis, provider concerns about CPM validity and the interpretation of CPM predictions may limit their clinical use.Providers felt that CPMs may be most useful for resource allocation, triage, research, or educational purposes for COVID-19.Several survivors of COVID-19 and their surrogates felt that CPMs would have been informative and may have aided them in making COVID-19 treatment decisions, while others felt the data would not have applied to them.

Importance: Anthracycline-containing regimens are highly effective for diffuse large B-cell lymphoma (DLBCL); however, patients with preexisting heart failure (HF) may be less likely to receive anthracyclines and may be at higher risk of lymphoma mortality., Objective: To assess the prevalence of preexisting HF in older patients with DLBCL and its association with treatment patterns and outcomes., Design, Setting, and Participants: This longitudinal cohort study used data from the Surveillance, Epidemiology, and End Results (SEER)-Medicare registry from 1999 to 2016. The SEER registry is a system of population-based cancer registries, capturing more than 25% of the US population. Linkage to Medicare offers additional information from billing claims. This study included individuals 65 years and older with newly diagnosed DLBCL from 2000 to 2015 with Medicare Part A or B continuously in the year prior to lymphoma diagnosis. Data were analyzed from September 2020 to December 2022., Exposures: Preexisting HF in the year prior to DLBCL diagnosis ascertained from billing codes required one of the following: (1) 1 primary inpatient discharge diagnosis, (2) 2 outpatient diagnoses, (3) 3 secondary inpatient discharge diagnoses, (4) 3 emergency department diagnoses, or (5) 2 secondary inpatient discharge diagnoses plus 1 outpatient diagnosis., Main Outcomes and Measures: The primary outcome was anthracycline-based treatment. The secondary outcomes were (1) cardioprotective medications and (2) cause-specific mortality. The associations between preexisting HF and cancer treatment were estimated using multivariable logistic regression. The associations between preexisting HF and cause-specific mortality were evaluated using cause-specific Cox proportional hazards models with adjustment for comorbidities and cancer treatment., Results: Of 30 728 included patients with DLBCL, 15 474 (50.4%) were female, and the mean (SD) age was 77.8 (7.2) years. Preexisting HF at lymphoma diagnosis was present in 4266 patients (13.9%). Patients with preexisting HF were less likely to be treated with an anthracycline (odds ratio, 0.55; 95% CI, 0.49-0.61). Among patients with preexisting HF who received an anthracycline, dexrazoxane or liposomal doxorubicin were used in 78 of 1119 patients (7.0%). One-year lymphoma mortality was 41.8% (95% CI, 40.5-43.2) with preexisting HF and 29.6% (95% CI, 29.0%-30.1%) without preexisting HF. Preexisting HF was associated with higher lymphoma mortality in models adjusting for baseline and time-varying treatment factors (hazard ratio, 1.24; 95% CI, 1.18-1.31)., Conclusions and Relevance: In this study, preexisting HF in patients with newly diagnosed DLBCL was common and was associated with lower use of anthracyclines and lower use of any chemotherapy. Trials are needed for this high-risk population.

Treatment effects are often anticipated to vary across groups of patients with different baseline risk. The Predictive Approaches to Treatment Effect Heterogeneity (PATH) statement focused on baseline risk as a robust predictor of treatment effect and provided guidance on risk-based assessment of treatment effect heterogeneity in a randomized controlled trial. The aim of this study is to extend this approach to the observational setting using a standardized scalable framework. The proposed framework consists of five steps: (1) definition of the research aim, i.e., the population, the treatment, the comparator and the outcome(s) of interest; (2) identification of relevant databases; (3) development of a prediction model for the outcome(s) of interest; (4) estimation of relative and absolute treatment effect within strata of predicted risk, after adjusting for observed confounding; (5) presentation of the results. We demonstrate our framework by evaluating heterogeneity of the effect of thiazide or thiazide-like diuretics versus angiotensin-converting enzyme inhibitors on three efficacy and nine safety outcomes across three observational databases. We provide a publicly available R software package for applying this framework to any database mapped to the Observational Medical Outcomes Partnership Common Data Model. In our demonstration, patients at low risk of acute myocardial infarction receive negligible absolute benefits for all three efficacy outcomes, though they are more pronounced in the highest risk group, especially for acute myocardial infarction. Our framework allows for the evaluation of differential treatment effects across risk strata, which offers the opportunity to consider the benefit-harm trade-off between alternative treatments., (© 2023. The Author(s).)

Background: Baseline outcome risk can be an important determinant of absolute treatment benefit and has been used in guidelines for "personalizing" medical decisions. We compared easily applicable risk-based methods for optimal prediction of individualized treatment effects., Methods: We simulated RCT data using diverse assumptions for the average treatment effect, a baseline prognostic index of risk, the shape of its interaction with treatment (none, linear, quadratic or non-monotonic), and the magnitude of treatment-related harms (none or constant independent of the prognostic index). We predicted absolute benefit using: models with a constant relative treatment effect; stratification in quarters of the prognostic index; models including a linear interaction of treatment with the prognostic index; models including an interaction of treatment with a restricted cubic spline transformation of the prognostic index; an adaptive approach using Akaike's Information Criterion. We evaluated predictive performance using root mean squared error and measures of discrimination and calibration for benefit., Results: The linear-interaction model displayed optimal or close-to-optimal performance across many simulation scenarios with moderate sample size (N = 4,250; ~ 785 events). The restricted cubic splines model was optimal for strong non-linear deviations from a constant treatment effect, particularly when sample size was larger (N = 17,000). The adaptive approach also required larger sample sizes. These findings were illustrated in the GUSTO-I trial., Conclusions: An interaction between baseline risk and treatment assignment should be considered to improve treatment effect predictions., (© 2023. The Author(s).)

Background: Health-related quality of life metrics evaluate treatments in ways that matter to patients, so are often included in randomised clinical trials (hereafter trials). Multimorbidity, where individuals have 2 or more conditions, is negatively associated with quality of life. However, whether multimorbidity predicts change over time or modifies treatment effects for quality of life is unknown. Therefore, clinicians and guideline developers are uncertain about the applicability of trial findings to people with multimorbidity. We examined whether comorbidity count (higher counts indicating greater multimorbidity) (i) is associated with quality of life at baseline; (ii) predicts change in quality of life over time; and/or (iii) modifies treatment effects on quality of life., Methods and Findings: Included trials were registered on the United States trials registry for selected index medical conditions and drug classes, phase 2/3, 3 or 4, had ≥300 participants, a nonrestrictive upper age limit, and were available on 1 of 2 trial repositories on 21 November 2016 and 18 May 2018, respectively. Of 124 meeting these criteria, 56 trials (33,421 participants, 16 index conditions, and 23 drug classes) collected a generic quality of life outcome measure (35 EuroQol-5 dimension (EQ-5D), 31 36-item short form survey (SF-36) with 10 collecting both). Blinding and completeness of follow up were examined for each trial. Using trials where individual participant data (IPD) was available from 2 repositories, a comorbidity count was calculated from medical history and/or prescriptions data. Linear regressions were fitted for the association between comorbidity count and (i) quality of life at baseline; (ii) change in quality of life during trial follow up; and (iii) treatment effects on quality of life. These results were then combined in Bayesian linear models. Posterior samples were summarised via the mean, 2.5th and 97.5th percentiles as credible intervals (95% CI) and via the proportion with values less than 0 as the probability (PBayes) of a negative association. All results are in standardised units (obtained by dividing the EQ-5D/SF-36 estimates by published population standard deviations). Per additional comorbidity, adjusting for age and sex, across all index conditions and treatment comparisons, comorbidity count was associated with lower quality of life at baseline and with a decline in quality of life over time (EQ-5D -0.02 [95% CI -0.03 to -0.01], PBayes > 0.999). Associations were similar, but with wider 95% CIs crossing the null for SF-36-PCS and SF-36-MCS (-0.05 [-0.10 to 0.01], PBayes = 0.956 and -0.05 [-0.10 to 0.01], PBayes = 0.966, respectively). Importantly, there was no evidence of any interaction between comorbidity count and treatment efficacy for either EQ-5D or SF-36 (EQ-5D -0.0035 [95% CI -0.0153 to -0.0065], PBayes = 0.746; SF-36-MCS (-0.0111 [95% CI -0.0647 to 0.0416], PBayes = 0.70 and SF-36-PCS -0.0092 [95% CI -0.0758 to 0.0476], PBayes = 0.631., Conclusions: Treatment effects on quality of life did not differ by multimorbidity (measured via a comorbidity count) at baseline-for the medical conditions studied, types and severity of comorbidities and level of quality of life at baseline, suggesting that evidence from clinical trials is likely to be applicable to settings with (at least modestly) higher levels of comorbidity., Trial Registration: A prespecified protocol was registered on PROSPERO (CRD42018048202)., Competing Interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests. NJW has received honoraria for training and masterclasses from: Association of British Pharmaceutical Industries, Campbell Ireland, Centre for Global Development, NICE International and NICE Scientific Advice, All Wales Therapeutics and Toxicology Centre, University of Leuven. NJW has delivered training for Takeda, ICON plc, and University of Galway for which payment was made to her institution. DM has received funding for this work from the Wellcome Trust., (Copyright: © 2023 Butterly et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Background Covert cerebrovascular disease (CCD) has been shown to be associated with dementia in population-based studies with magnetic resonance imaging (MRI) screening, but dementia risk associated with incidentally discovered CCD is not known. Methods and Results Individuals aged ≥50 years enrolled in the Kaiser Permanente Southern California health system receiving head computed tomography (CT) or MRI for nonstroke indications from 2009 to 2019, without prior ischemic stroke/transient ischemic attack, dementia/Alzheimer disease, or visit reason/scan indication suggestive of cognitive decline or stroke were included. Natural language processing identified incidentally discovered covert brain infarction (id-CBI) and white matter disease (id-WMD) on the neuroimage report; white matter disease was characterized as mild, moderate, severe, or undetermined. We estimated risk of dementia associated with id-CBI and id-WMD. Among 241 050 qualified individuals, natural language processing identified 69 931 (29.0%) with id-WMD and 11 328 (4.7%) with id-CBI. Dementia incidence rates (per 1000 person-years) were 23.5 (95% CI, 22.9-24.0) for patients with id-WMD, 29.4 (95% CI, 27.9-31.0) with id-CBI, and 6.0 (95% CI, 5.8-6.2) without id-CCD. The association of id-WMD with future dementia was stronger in younger (aged <70 years) versus older (aged ≥70 years) patients and for CT- versus MRI-discovered lesions. For patients with versus without id-WMD on CT, the adjusted HR was 2.87 (95% CI, 2.58-3.19) for older and 1.87 (95% CI, 1.79-1.95) for younger patients. For patients with versus without id-WMD on MRI, the adjusted HR for dementia risk was 2.28 (95% CI, 1.99-2.62) for older and 1.48 (95% CI, 1.32-1.66) for younger patients. The adjusted HR for id-CBI was 2.02 (95% CI, 1.70-2.41) for older and 1.22 (95% CI, 1.15-1.30) for younger patients for either modality. Dementia risk was strongly correlated with id-WMD severity; adjusted HRs compared with patients who were negative for id-WMD by MRI ranged from 1.41 (95% CI, 1.25-1.60) for those with mild disease on MRI to 4.11 (95% CI, 3.58-4.72) for those with severe disease on CT. Conclusions Incidentally discovered CCD is common and associated with a high risk of dementia, representing an opportunity for prevention. The association is strengthened when discovered at younger age, by increasing id-WMD severity, and when id-WMD is detected by CT scan rather than MRI.

Background: Supporting decisions for patients who present to the emergency department (ED) with COVID-19 requires accurate prognostication. We aimed to evaluate prognostic models for predicting outcomes in hospitalized patients with COVID-19, in different locations and across time., Methods: We included patients who presented to the ED with suspected COVID-19 and were admitted to 12 hospitals in the New York City (NYC) area and 4 large Dutch hospitals. We used second-wave patients who presented between September and December 2020 (2137 and 3252 in NYC and the Netherlands, respectively) to evaluate models that were developed on first-wave patients who presented between March and August 2020 (12,163 and 5831). We evaluated two prognostic models for in-hospital death: The Northwell COVID-19 Survival (NOCOS) model was developed on NYC data and the COVID Outcome Prediction in the Emergency Department (COPE) model was developed on Dutch data. These models were validated on subsequent second-wave data at the same site (temporal validation) and at the other site (geographic validation). We assessed model performance by the Area Under the receiver operating characteristic Curve (AUC), by the E-statistic, and by net benefit., Results: Twenty-eight-day mortality was considerably higher in the NYC first-wave data (21.0%), compared to the second-wave (10.1%) and the Dutch data (first wave 10.8%; second wave 10.0%). COPE discriminated well at temporal validation (AUC 0.82), with excellent calibration (E-statistic 0.8%). At geographic validation, discrimination was satisfactory (AUC 0.78), but with moderate over-prediction of mortality risk, particularly in higher-risk patients (E-statistic 2.9%). While discrimination was adequate when NOCOS was tested on second-wave NYC data (AUC 0.77), NOCOS systematically overestimated the mortality risk (E-statistic 5.1%). Discrimination in the Dutch data was good (AUC 0.81), but with over-prediction of risk, particularly in lower-risk patients (E-statistic 4.0%). Recalibration of COPE and NOCOS led to limited net benefit improvement in Dutch data, but to substantial net benefit improvement in NYC data., Conclusions: NOCOS performed moderately worse than COPE, probably reflecting unique aspects of the early pandemic in NYC. Frequent updating of prognostic models is likely to be required for transportability over time and space during a dynamic pandemic., (© 2022. The Author(s).)

Clinical prognostic models can assist patient care decisions. However, their performance can drift over time and location, necessitating model monitoring and updating. Despite rapid and significant changes during the pandemic, prognostic models for COVID-19 patients do not currently account for these drifts. We develop a framework for continuously monitoring and updating prognostic models and apply it to predict 28-day survival in COVID-19 patients. We use demographic, laboratory, and clinical data from electronic health records of 34912 hospitalized COVID-19 patients from March 2020 until May 2022 and compare three modeling methods. Model calibration performance drift is immediately detected with minor fluctuations in discrimination. The overall calibration on the prospective validation cohort is significantly improved when comparing the dynamically updated models against their static counterparts. Our findings suggest that, using this framework, models remain accurate and well-calibrated across various waves, variants, race and sex and yield positive net-benefits., (© 2022. The Author(s).)

Importance: The Patent Foramen Ovale (PFO)-Associated Stroke Causal Likelihood classification system combines information regarding noncardiac patient features (vascular risk factors, infarct topography) and PFO features (shunt size and presence of atrial septal aneurysm [ASA]) to classify patients into 3 validated categories of responsiveness to treatment with PFO closure. However, the distinctive associations of shunt size and ASA, alone and in combination, have not been completely delineated., Objective: To evaluate the association of PFO closure with stroke recurrence according to shunt size and/or the presence of an ASA., Design, Setting, and Participants: Pooled individual patient data from 6 randomized clinical trials conducted from February 2000 to October 2017 that compared PFO closure with medical therapy. Patients in North America, Europe, Australia, Brazil, and South Korea with PFO-associated stroke were included. Analysis was completed in January 2022., Exposures: Transcatheter PFO closure plus antithrombotic therapy vs antithrombotic therapy alone, stratified into 4 groups based on the combination of 2 features: small vs large PFO shunt size and the presence or absence of an ASA., Main Outcomes and Measures: Recurrent ischemic stroke., Results: A total of 121 recurrent ischemic strokes occurred in the pooled 3740 patients (mean [SD] age, 45 [10] years; 1682 [45%] female) during a median (IQR) follow-up of 57 (23.7-63.8) months. Treatment with PFO closure was associated with reduced risk for recurrent ischemic stroke (adjusted hazard ratio [aHR], 0.41 [95% CI, 0.28-0.60]; P < .001). The reduction in hazard for recurrent stroke was greater for patients with both a large shunt and an ASA (aHR, 0.15 [95% CI, 0.06-0.35]) than for large shunt without ASA (aHR, 0.27 [95% CI, 0.14-0.56]), small shunt with ASA (aHR, 0.36 [95% CI, 0.17-0.78]), and small shunt without ASA (aHR, 0.68 [95% CI, 0.41-1.13]) (interaction P = .02). At 2 years, the absolute risk reduction of recurrent stroke was greater (5.5% [95% CI, 2.7-8.3]) in patients with large shunt and ASA than for patients in the other 3 categories (1.0% for all)., Conclusions and Relevance: Patients with both a large shunt and an ASA showed a substantially greater beneficial association with PFO closure than patients with large shunt alone, patients with small shunt and ASA, and patients with neither large shunt nor ASA. These findings, combined with other patient features, may inform shared patient-clinician decision-making.

Competing Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Background This study aimed to compare percutaneous left atrial appendage occlusion (LAAO) with non-vitamin K antagonist oral anticoagulants among patients with atrial fibrillation. Methods and Results Using a US administrative database, 562 850 patients with atrial fibrillation were identified, among whom 8397 were treated with LAAO and 554 453 were treated with non-vitamin K antagonist oral anticoagulants between March 13, 2015 and December 31, 2018. Propensity score overlap weighting was used to balance baseline characteristics. The primary outcome was a composite end point of ischemic stroke or systemic embolism, major bleeding, and all-cause mortality. The mean age was 76.4±7.6 years; 280 097 (49.8%) were female. Mean follow-up was 1.5±1.0 years. LAAO was associated with no significant difference in the risk of the primary composite end point (hazard ratio [HR], 0.93 [0.84-1.03]), or the secondary outcomes including ischemic stroke/systemic embolism (HR, 1.07 [0.81-1.41]), and intracranial bleeding (HR, 1.08 [0.72-1.61]). LAAO was associated with a higher risk of major bleeding (HR, 1.22 [1.05-1.42], P =0.01) and a lower risk of mortality (HR, 0.73 [0.64-0.84], P <0.001). The lower risk of mortality associated with LAAO was most pronounced in patients with a prior history of intracranial bleeding. Conclusions In comparison to non-vitamin K antagonist oral anticoagulants, LAAO was associated with no significant difference in the risk of the composite outcome and a lower risk of mortality, which suggests LAAO might be a reasonable option in select patients with atrial fibrillation. The observation of higher bleeding risk associated with LAAO highlights the need to optimize postprocedural antithrombotic regimens as well as systematic efforts to assess and address bleeding predispositions.

Objective: This study aimed to examine the relationship between covert cerebrovascular disease, comprised of covert brain infarction and white matter disease, discovered incidentally in routine care, and subsequent Parkinson disease., Methods: Patients were ≥50 years and received neuroimaging for non-stroke indications in the Kaiser Permanente Southern California system from 2009 to 2019. Natural language processing identified incidentally discovered covert brain infarction and white matter disease and classified white matter disease severity. The Parkinson disease outcome was defined as 2 ICD diagnosis codes., Results: 230,062 patients were included (median follow-up 3.72 years). A total of 1,941 Parkinson disease cases were identified (median time-to-event 2.35 years). Natural language processing identified covert cerebrovascular disease in 70,592 (30.7%) patients, 10,622 (4.6%) with covert brain infarction and 65,814 (28.6%) with white matter disease. After adjustment for known risk factors, white matter disease was associated with Parkinson disease (hazard ratio 1.67 [95%CI, 1.44, 1.93] for patients <70 years and 1.33 [1.18, 1.50] for those ≥70 years). Greater severity of white matter disease was associated with increased incidence of Parkinson disease(/1,000 person-years), from 1.52 (1.43, 1.61) in patients without white matter disease to 4.90 (3.86, 6.13) in those with severe disease. Findings were robust when more specific definitions of Parkinson disease were used. Covert brain infarction was not associated with Parkinson disease (adjusted hazard ratio = 1.05 [0.88, 1.24])., Interpretation: Incidentally discovered white matter disease was associated with subsequent Parkinson disease, an association strengthened with younger age and increased white matter disease severity. Incidentally discovered covert brain infarction did not appear to be associated with subsequent Parkinson disease. ANN NEUROL 2022;92:620-630., (© 2022 American Neurological Association.)

Objective: To assess the frequency of silent brain infarcts (SBIs) and white matter disease (WMD) and associations with stroke risk factors (RFs) in a real-world population., Patients and Methods: This was an observational study of patients 50 years or older in the Kaiser Permanente Southern California health system from January 1, 2009, through June 30, 2019, with head computed tomography or magnetic resonance imaging for nonstroke indications and no history of stroke, transient ischemic attack, or dementia. A natural language processing (NLP) algorithm was applied to the electronic health record to identify individuals with reported SBIs or WMD. Multivariable Poisson regression estimated risk ratios of demographic characteristics, RFs, and scan modality on the presence of SBIs or WMD., Results: Among 262,875 individuals, the NLP identified 13,154 (5.0%) with SBIs and 78,330 (29.8%) with WMD. Stroke RFs were highly prevalent. Advanced age was strongly associated with increased risk of SBIs (adjusted relative risks [aRRs], 1.90, 3.23, and 4.72 for those aged in their 60s, 70s, and ≥80s compared with those in their 50s) and increased risk of WMD (aRRs, 1.79, 3.02, and 4.53 for those aged in their 60s, 70s, and ≥80s compared with those in their 50s). Magnetic resonance imaging was associated with a reduced risk of SBIs (aRR, 0.87; 95% CI, 0.83 to 0.91) and an increased risk of WMD (aRR, 2.86; 95% CI, 2.83 to 2.90). Stroke RFs had modest associations with increased risk of SBIs or WMD., Conclusion: An NLP algorithm can identify a large cohort of patients with incidentally discovered SBIs and WMD. Advanced age is strongly associated with incidentally discovered SBIs and WMD., (Copyright © 2021 Mayo Foundation for Medical Education and Research. Published by Elsevier Inc. All rights reserved.)

Background: Prediction modeling studies often have methodological limitations, which may compromise model performance in new patients and settings. We aimed to examine the relation between methodological quality of model development studies and their performance at external validation., Methods: We systematically searched for externally validated multivariable prediction models that predict functional outcome following moderate or severe traumatic brain injury. Risk of bias and applicability of development studies was assessed with the Prediction model Risk Of Bias Assessment Tool (PROBAST). Each model was rated for its presentation with sufficient detail to be used in practice. Model performance was described in terms of discrimination (AUC), and calibration. Delta AUC (dAUC) was calculated to quantify the percentage change in discrimination between development and validation for all models. Generalized estimation equations (GEE) were used to examine the relation between methodological quality and dAUC while controlling for clustering., Results: We included 54 publications, presenting ten development studies of 18 prediction models, and 52 external validation studies, including 245 unique validations. Two development studies (four models) were found to have low risk of bias (RoB). The other eight publications (14 models) showed high or unclear RoB. The median dAUC was positive in low RoB models (dAUC 8%, [IQR - 4% to 21%]) and negative in high RoB models (dAUC - 18%, [IQR - 43% to 2%]). The GEE showed a larger average negative change in discrimination for high RoB models (- 32% (95% CI: - 48 to - 15) and unclear RoB models (- 13% (95% CI: - 16 to - 10)) compared to that seen in low RoB models., Conclusion: Lower methodological quality at model development associates with poorer model performance at external validation. Our findings emphasize the importance of adherence to methodological principles and reporting guidelines in prediction modeling studies., (© 2022. The Author(s).)

Purpose : For patients diagnosed with ALS, dysphagia can result in aspiration, malnutrition, and mortality. The purpose of this study was to develop a clinical prediction model capable of identifying patients with ALS at imminent risk for developing swallowing complications. Methods : A retrospective cohort study using the Pooled Resource Open-Access ALS Clinical Trials Database (PRO-ACT) was conducted. After dividing the PRO-ACT database into development and validation cohorts with dysphagia defined from the ALS Functional Rating Scale (ALSFRS), a multivariable Cox proportional hazards regression model estimated the probability of dysphagia at 3, 6, and 12-months with subsequent evaluation of model discrimination and calibration. Results : With 2057 participants in the development cohort and 1891 in the validation cohort, the Cox model included 7 clinical variables: spinal-onset; bulbar, fine and gross motor ALSFRS subscale scores; respiratory impairment; functional progression rate; and time from diagnosis. The cumulative incidence of dysphagia was 18% at 3-months, 29% at 6-months, and 45% at 12-months. The mean predicted probability of dysphagia development ranged from 4.5% in the bottommost risk decile to 40% in the topmost decile at 3 months, 10%-72% at 6 months, and 25%-93% at 12 months. In the validation cohort, the model had good discrimination and calibration with an optimism corrected c-statistic of 0.70 and calibration slope of 0.96. Conclusions : The ALS dysphagia risk score can be used to identify patients with ALS at high risk for self-reported dysphagia development who would benefit from a comprehensive swallowing assessment and proactive dysphagia management strategies.

Background: Although randomized trials have established that coronary artery bypass grafting (CABG) is, on average, the most effective revascularization strategy compared with percutaneous coronary intervention (PCI) in patients with diabetes and multivessel disease (MVD), individual patients differ in many characteristics that can affect the benefits and harms of treatment. The FREEDOM (Future Revascularization Evaluation in Patients with Diabetes Mellitus) score was developed to predict different outcomes with CABG vs PCI on the basis of 8 patient characteristics and the smoking-treatment interaction., Objectives: This study aimed to assess the ability of the 5-year major adverse cardiovascular event (MACE) model to predict treatment benefit of CABG vs PCI in the SYNTAX (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery) and BEST (Bypass Surgery and Everolimus-Eluting Stent Implantation in the Treatment of Patients with Multivessel Coronary Artery Disease) trials., Methods: This study identified 702 patients with diabetes and MVD to mirror the FREEDOM participants. Discrimination was assessed by C-index, and calibration was assessed by calibration plots in the PCI and CABG arms, respectively. The ability of the FREEDOM score to predict treatment benefit of CABG vs PCI was assessed., Results: Overall, CABG was associated with a lower rate of 5-year MACE compared with PCI (12.4% vs 20.3%; log-rank P = 0.021) irrespective of a history of smoking (P interaction  = 0.975). Both discrimination and calibration were helpful in the PCI arm (C-index: 0.69; slope: 0.96, intercept: -0.24), but moderate in the CABG arm (C-index: 0.61; slope: 0.61; intercept: -0.53). The FREEDOM score showed some heterogeneity of treatment benefit., Conclusions: The FREEDOM score could identify some heterogeneity of treatment benefit of CABG vs PCI for 5-year MACE. Until further prospective validations are performed, these results should be taken into consideration when using the FREEDOM score in patients with diabetes and MVD. (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery [SYNTAX]; NCT00114972) (Bypass Surgery and Everolimus-Eluting Stent Implantation in the Treatment of Patients with Multivessel Coronary Artery Disease [BEST]; NCT00997828) (Future Revascularization Evaluation in Patients with Diabetes Mellitus [FREEDOM]; NCT00086450)., Competing Interests: Funding Support and Author Disclosures This work was supported by Boston Scientific Corporation (SYNTAX study, 0-5-year follow-up). The study funders had no role in trial design, data collection, data analyses, interpretation of the data, writing of the report, or making a decision to submit the manuscript for publication. Dr Serruys has received personal consulting fees from Biosensors, Medtronic, Micell, Sino Medical Sciences Technology, Philips Volcano, Xeltis, and Heartflow, outside the submitted work. Dr Farkouh has received research grants from Amgen, Novartis, and Novo Nordisk, outside the submitted work. Dr Spertus has ownership of the Seattle Angina Questionnaire; has received consulting fees from United Healthcare, Merck, Bayer, Novartis, Janssen, Myokardia, and Janssen; has received a research grant from Abbott Vascular; and has served on the Board of Directors of Blue Cross Blue Shield of Kansas City, all outside the submitted work. Dr Cohen has received institutional research grants and personal fees from Medtronic, Boston Scientific, and Abbott Vascular, outside the submitted work. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose., (Copyright © 2022 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.)

Background: While clinical prediction models (CPMs) are used increasingly commonly to guide patient care, the performance and clinical utility of these CPMs in new patient cohorts is poorly understood., Methods: We performed 158 external validations of 104 unique CPMs across 3 domains of cardiovascular disease (primary prevention, acute coronary syndrome, and heart failure). Validations were performed in publicly available clinical trial cohorts and model performance was assessed using measures of discrimination, calibration, and net benefit. To explore potential reasons for poor model performance, CPM-clinical trial cohort pairs were stratified based on relatedness, a domain-specific set of characteristics to qualitatively grade the similarity of derivation and validation patient populations. We also examined the model-based C-statistic to assess whether changes in discrimination were because of differences in case-mix between the derivation and validation samples. The impact of model updating on model performance was also assessed., Results: Discrimination decreased significantly between model derivation (0.76 [interquartile range 0.73-0.78]) and validation (0.64 [interquartile range 0.60-0.67], P <0.001), but approximately half of this decrease was because of narrower case-mix in the validation samples. CPMs had better discrimination when tested in related compared with distantly related trial cohorts. Calibration slope was also significantly higher in related trial cohorts (0.77 [interquartile range, 0.59-0.90]) than distantly related cohorts (0.59 [interquartile range 0.43-0.73], P =0.001). When considering the full range of possible decision thresholds between half and twice the outcome incidence, 91% of models had a risk of harm (net benefit below default strategy) at some threshold; this risk could be reduced substantially via updating model intercept, calibration slope, or complete re-estimation., Conclusions: There are significant decreases in model performance when applying cardiovascular disease CPMs to new patient populations, resulting in substantial risk of harm. Model updating can mitigate these risks. Care should be taken when using CPMs to guide clinical decision-making.

Objective: To develop an electronic health record (EHR)-based risk tool that provides point-of-care estimates of diabetes risk to support targeting interventions to patients most likely to benefit., Patients and Methods: A risk prediction model was developed and validated in a large observational database of patients with an index visit date between January 1, 2012, and December 31, 2016, with treatment effect estimates from risk-based reanalysis of clinical trial data. The risk model development cohort included 1.1 million patients with prediabetes from the OptumLabs Data Warehouse (OLDW); the validation cohort included a distinct sample of 1.1 million patients in OLDW. The randomly assigned clinical trial cohort included 3081 people from the Diabetes Prevention Program (DPP) study., Results: Eleven variables reliably obtainable from the EHR were used to predict diabetes risk. This model validated well in the OLDW (C statistic = 0.76; observed 3-year diabetes rate was 1.8% (95% confidence interval [CI], 1.7 to 1.9) in the lowest-risk quarter and 19.6% (19.4 to 19.8) in the highest-risk quarter). In the DPP, the hazard ratio (HR) for lifestyle modification was constant across all levels of risk (HR, 0.43; 95% CI, 0.35 to 0.53), whereas the HR for metformin was highly risk dependent (HR, 1.1; 95% CI, 0.61 to 2.0 in the lowest-risk quarter vs HR, 0.45; 95% CI, 0.35 to 0.59 in the highest-risk quarter). Fifty-three percent of the benefits of population-wide dissemination of the DPP lifestyle modification and 73% of the benefits of population-wide metformin therapy can be obtained by targeting the highest-risk quarter of patients., Conclusion: The Tufts-Predictive Analytics and Comparative Effectiveness DPP Risk model is an EHR-compatible tool that might support targeted diabetes prevention to more efficiently realize the benefits of the DPP interventions., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)

A randomized trial of the GRAIL Galleri TM multi-cancer screening test is being planned for the National Health Service in England, and will have 140,000 healthy participants aged 50-79: 70,000 exposed to screening and 70,000 unexposed. The test reportedly detects 50 different cancers and is expected to reduce all-cancer mortality by approximately 25%. Given this effect size-and that cancer deaths constitute a large fraction of all deaths-the trial is sufficiently large to test the effect on all-cause mortality. Because most patients believe cancer screening "saves lives", the GRAIL/National Health Service collaboration could set the evaluation standard for multi-cancer screening.

Importance: Patent foramen ovale (PFO)-associated strokes comprise approximately 10% of ischemic strokes in adults aged 18 to 60 years. While device closure decreases stroke recurrence risk overall, the best treatment for any individual is often unclear., Objective: To evaluate heterogeneity of treatment effect of PFO closure on stroke recurrence based on previously developed scoring systems., Design, Setting, and Participants: Investigators for the Systematic, Collaborative, PFO Closure Evaluation (SCOPE) Consortium pooled individual patient data from all 6 randomized clinical trials that compared PFO closure plus medical therapy vs medical therapy alone in patients with PFO-associated stroke, and included a total of 3740 participants. The trials were conducted worldwide from 2000 to 2017., Exposures: PFO closure plus medical therapy vs medical therapy alone. Subgroup analyses used the Risk of Paradoxical Embolism (RoPE) Score (a 10-point scoring system in which higher scores reflect younger age and the absence of vascular risk factors) and the PFO-Associated Stroke Causal Likelihood (PASCAL) Classification System, which combines the RoPE Score with high-risk PFO features (either an atrial septal aneurysm or a large-sized shunt) to classify patients into 3 categories of causal relatedness: unlikely, possible, and probable., Main Outcomes and Measures: Ischemic stroke., Results: Over a median follow-up of 57 months (IQR, 24-64), 121 outcomes occurred in 3740 patients. The annualized incidence of stroke with medical therapy was 1.09% (95% CI, 0.88%-1.36%) and with device closure was 0.47% (95% CI, 0.35%-0.65%) (adjusted hazard ratio [HR], 0.41 [95% CI, 0.28-0.60]). The subgroup analyses showed statistically significant interaction effects. Patients with low vs high RoPE Score had HRs of 0.61 (95% CI, 0.37-1.00) and 0.21 (95% CI, 0.11-0.42), respectively (P for interaction = .02). Patients classified as unlikely, possible, and probable using the PASCAL Classification System had HRs of 1.14 (95% CI, 0.53-2.46), 0.38 (95% CI, 0.22-0.65), and 0.10 (95% CI, 0.03-0.35), respectively (P for interaction = .003). The 2-year absolute risk reduction was -0.7% (95% CI, -4.0% to 2.6%), 2.1% (95% CI, 0.6%-3.6%), and 2.1% (95% CI, 0.9%-3.4%) in the unlikely, possible, and probable PASCAL categories, respectively. Device-associated adverse events were generally higher among patients classified as unlikely; the absolute risk increases in atrial fibrillation beyond day 45 after randomization with a device were 4.41% (95% CI, 1.02% to 7.80%), 1.53% (95% CI, 0.33% to 2.72%), and 0.65% (95% CI, -0.41% to 1.71%) in the unlikely, possible, and probable PASCAL categories, respectively., Conclusions and Relevance: Among patients aged 18 to 60 years with PFO-associated stroke, risk reduction for recurrent stroke with device closure varied across groups classified by their probabilities that the stroke was causally related to the PFO. Application of this classification system has the potential to guide individualized decision-making.

Objective: To assess whether the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and a shorter version of this tool can identify clinical prediction models (CPMs) that perform poorly at external validation., Study Design and Setting: We evaluated risk of bias (ROB) on 102 CPMs from the Tufts CPM Registry, comparing PROBAST to a short form consisting of six PROBAST items anticipated to best identify high ROB. We then applied the short form to all CPMs in the Registry with at least 1 validation (n=556) and assessed the change in discrimination (dAUC) in external validation cohorts (n=1,147)., Results: PROBAST classified 98/102 CPMS as high ROB. The short form identified 96 of these 98 as high ROB (98% sensitivity), with perfect specificity. In the full CPM registry, 527 of 556 CPMs (95%) were classified as high ROB, 20 (3.6%) low ROB, and 9 (1.6%) unclear ROB. Only one model with unclear ROB was reclassified to high ROB after full PROBAST assessment of all low and unclear ROB models. Median change in discrimination was significantly smaller in low ROB models (dAUC -0.9%, IQR -6.2-4.2%) compared to high ROB models (dAUC -11.7%, IQR -33.3-2.6%; P<0.001)., Conclusion: High ROB is pervasive among published CPMs. It is associated with poor discriminative performance at validation, supporting the application of PROBAST or a shorter version in CPM reviews., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)

Background and Objectives: Silent cerebrovascular disease (SCD), comprising silent brain infarction (SBI) and white matter disease (WMD), is commonly found incidentally on neuroimaging scans obtained in routine clinical care. Their prognostic significance is not known. We aimed to estimate the incidence of and risk increase in future stroke in patients with incidentally discovered SCD., Methods: Patients in the Kaiser Permanente Southern California (KPSC) health system aged ≥50 years, without prior ischemic stroke, transient ischemic attack (TIA), or dementia/Alzheimer disease receiving a head CT or MRI between 2009 and 2019 were included. SBI and WMD were identified by natural language processing (NLP) from the neuroimage report., Results: Among 262,875 individuals receiving neuroimaging, NLP identified 13,154 (5.0%) with SBI and 78,330 (29.8%) with WMD. The incidence of future stroke was 32.5 (95% confidence interval [CI] 31.1, 33.9) per 1,000 patient-years for patients with SBI: 19.3 (95% CI 18.9, 19.8) for patients with WMD and 6.8 (95% CI 6.7, 7.0) for patients without SCD. The crude hazard ratio (HR) associated with SBI was 3.40 (95% CI 3.25 to 3.56) and for WMD 2.63 (95% CI 2.54 to 2.71). With MRI-discovered SBI, the adjusted HR was 2.95 (95% CI 2.53 to 3.44) for those <65 years of age and 2.15 (95% CI 1.91 to 2.41) for those ≥65. With CT scan, the adjusted HR was 2.48 (95% CI 2.19 to 2.81) for those <65 and 1.81 (95% CI 1.71 to 1.91) for those ≥65. The adjusted HR associated with a finding of WMD was 1.76 (95% CI 1.69 to 1.82) and was not modified by age or imaging modality., Discussion: Incidentally discovered SBI and WMD are common and associated with increased risk of subsequent symptomatic stroke, representing an important opportunity for stroke prevention., (© 2021 American Academy of Neurology.)

Background: The SYNTAX score II 2020 (SSII-2020) was derived from cross correlation and externally validated in randomized trials to predict death and major adverse cardiac and cerebrovascular events (MACE) following percutaneous coronary intervention (PCI) and coronary artery bypass grafting (CABG) in patients with 3-vessel disease (3VD) and/or left main coronary artery disease (LMCAD)., Objectives: The authors aimed to investigate the SSII-2020's value in identifying the safest modality of revascularization in a non-randomized setting., Methods: Five-year mortality and MACE were assessed in 7,362 patients with 3VD and/or LMCAD enrolled in a Japanese PCI/CABG registry. The discriminative abilities of the SSII-2020 were assessed using Harrell's C statistic. Agreement between observed and predicted event rates following PCI or CABG and treatment benefit (absolute risk difference [ARD]) for these outcomes were assessed by calibration plots., Results: The SSII-2020 for 5-year mortality well predicted the prognosis after PCI and CABG (C-index = 0.72, intercept = -0.11, slope = 0.92). When patients were grouped according to the predicted 5-year mortality ARD, <4.5% (equipoise of PCI and CABG) and ≥4.5% (CABG better), the observed mortality rates after PCI and CABG were not significantly different in patients with lower predicted ARD (observed ARD: 2.1% [95% CI: -0.4% to 4.4%]), and the significant difference in survival in favor of CABG was observed in patients with higher predicted ARD (observed ARD: 9.7% [95% CI: 6.1%-13.3%]). For MACE, the SSII-2020 could not recommend a specific treatment with sufficient accuracy., Conclusions: The SSII-2020 for predicting 5-year death has the potential to support decision making on revascularization in patients with 3VD and/or LMCAD., Competing Interests: Funding Support and Author Disclosures Dr Hara has received a grant for studying overseas from Japanese Circulation Society, a grant-in-Aid for JSPS Fellows, and a grant from Fukuda Foundation for Medical Technology. Dr Shiomi has received personal fees from Abbott Vascular, Boston Scientific, and Daiichi-Sankyo, outside of the submitted work. Dr Serruys has received personal fees from Biosensors, Micel Technologies, Sinomedical Sciences Technology, Philips/Volcano, Xeltis, and HeartFlow, outside of the submitted work. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose., (Copyright © 2021 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.)

Objectives: Develop simple and valid models for predicting mortality and need for intensive care unit (ICU) admission in patients who present at the emergency department (ED) with suspected COVID-19., Design: Retrospective., Setting: Secondary care in four large Dutch hospitals., Participants: Patients who presented at the ED and were admitted to hospital with suspected COVID-19. We used 5831 first-wave patients who presented between March and August 2020 for model development and 3252 second-wave patients who presented between September and December 2020 for model validation., Outcome Measures: We developed separate logistic regression models for in-hospital death and for need for ICU admission, both within 28 days after hospital admission. Based on prior literature, we considered quickly and objectively obtainable patient characteristics, vital parameters and blood test values as predictors. We assessed model performance by the area under the receiver operating characteristic curve (AUC) and by calibration plots., Results: Of 5831 first-wave patients, 629 (10.8%) died within 28 days after admission. ICU admission was fully recorded for 2633 first-wave patients in 2 hospitals, with 214 (8.1%) ICU admissions within 28 days. A simple model-COVID outcome prediction in the emergency department (COPE)-with age, respiratory rate, C reactive protein, lactate dehydrogenase, albumin and urea captured most of the ability to predict death. COPE was well calibrated and showed good discrimination for mortality in second-wave patients (AUC in four hospitals: 0.82 (95% CI 0.78 to 0.86); 0.82 (95% CI 0.74 to 0.90); 0.79 (95% CI 0.70 to 0.88); 0.83 (95% CI 0.79 to 0.86)). COPE was also able to identify patients at high risk of needing ICU admission in second-wave patients (AUC in two hospitals: 0.84 (95% CI 0.78 to 0.90); 0.81 (95% CI 0.66 to 0.95))., Conclusions: COPE is a simple tool that is well able to predict mortality and need for ICU admission in patients who present to the ED with suspected COVID-19 and may help patients and doctors in decision making., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: There are many clinical prediction models (CPMs) available to inform treatment decisions for patients with cardiovascular disease. However, the extent to which they have been externally tested, and how well they generally perform has not been broadly evaluated., Methods: A SCOPUS citation search was run on March 22, 2017 to identify external validations of cardiovascular CPMs in the Tufts Predictive Analytics and Comparative Effectiveness CPM Registry. We assessed the extent of external validation, performance heterogeneity across databases, and explored factors associated with model performance, including a global assessment of the clinical relatedness between the derivation and validation data., Results: We identified 2030 external validations of 1382 CPMs. Eight hundred seven (58%) of the CPMs in the Registry have never been externally validated. On average, there were 1.5 validations per CPM (range, 0-94). The median external validation area under the receiver operating characteristic curve was 0.73 (25th-75th percentile [interquartile range (IQR)], 0.66-0.79), representing a median percent decrease in discrimination of -11.1% (IQR, -32.4% to +2.7%) compared with performance on derivation data. 81% (n=1333) of validations reporting area under the receiver operating characteristic curve showed discrimination below that reported in the derivation dataset. 53% (n=983) of the validations report some measure of CPM calibration. For CPMs evaluated more than once, there was typically a large range of performance. Of 1702 validations classified by relatedness, the percent change in discrimination was -3.7% (IQR, -13.2 to 3.1) for closely related validations (n=123), -9.0 (IQR, -27.6 to 3.9) for related validations (n=862), and -17.2% (IQR, -42.3 to 0) for distantly related validations (n=717; P <0.001)., Conclusions: Many published cardiovascular CPMs have never been externally validated, and for those that have, apparent performance during development is often overly optimistic. A single external validation appears insufficient to broadly understand the performance heterogeneity across different settings.

Introduction/aims: Dysphagia worsens mortality and quality of life for persons diagnosed with amyotrophic lateral sclerosis (ALS), yet our understanding of its incidence and timing remains limited. In this study we sought to estimate dysphagia incidence and dysphagia-free survival over time., Methods: Using data from the Pooled Resource Open-Access ALS Clinical Trials Database, we compared characteristics of persons with and without dysphagia upon study entry. To account for competing mortality risk, we used Kaplan-Meier curves to estimate the cumulative incidence of dysphagia and the median number of days until the development of dysphagia or death in those without dysphagia at study entry., Results: Patients with dysphagia upon study entry were more likely to have bulbar onset and had faster rates of functional decline and shorter diagnostic delays. The cumulative incidence of new-onset dysphagia was 44% at 1 year and 64% at 2 years after trial enrollment for those with spinal onset, and 85% and 92% for those with bulbar onset. The median duration of dysphagia-free survival after trial enrollment was 11.5 months for those with spinal onset and 3.2 months for those with bulbar onset., Discussion: Our findings underscore the high risk for dysphagia development and support the need for early dysphagia referral and evaluation to minimize the risk of serious dysphagia-related complications., (© 2021 Wiley Periodicals LLC.)

Objective: Approximately 84 million people in the USA have pre-diabetes, but only a fraction of them receive proven effective therapies to prevent type 2 diabetes. We estimated the value of prioritizing individuals at highest risk of progression to diabetes for treatment, compared to non-targeted treatment of individuals meeting inclusion criteria for the Diabetes Prevention Program (DPP)., Methods: Using microsimulation to project outcomes in the DPP trial population, we compared two interventions to usual care: (1) lifestyle modification and (2) metformin administration. For each intervention, we compared targeted and non-targeted strategies, assuming either limited or unlimited program capacity. We modeled the individualized risk of developing diabetes and projected diabetic outcomes to yield lifetime costs and quality-adjusted life expectancy, from which we estimated net monetary benefits (NMB) for both lifestyle and metformin versus usual care., Results: Compared to usual care, lifestyle modification conferred positive benefits and reduced lifetime costs for all eligible individuals. Metformin's NMB was negative for the lowest population risk quintile. By avoiding use when costs outweighed benefits, targeted administration of metformin conferred a benefit of $500 per person. If only 20% of the population could receive treatment, when prioritizing individuals based on diabetes risk, rather than treating a 20% random sample, the difference in NMB ranged from $14,000 to $20,000 per person., Conclusions: Targeting active diabetes prevention to patients at highest risk could improve health outcomes and reduce costs compared to providing the same intervention to a similar number of patients with pre-diabetes without targeted selection.

Background: There are numerous barriers to identifying patients with silent brain infarcts (SBIs) and white matter disease (WMD) in routine clinical care. A natural language processing (NLP) algorithm may identify patients from neuroimaging reports, but it is unclear if these reports contain reliable information on these findings., Methods: Four radiology residents reviewed 1000 neuroimaging reports (RI) of patients age > 50 years without clinical histories of stroke, TIA, or dementia for the presence, acuity, and location of SBIs, and the presence and severity of WMD. Four neuroradiologists directly reviewed a subsample of 182 images (DR). An NLP algorithm was developed to identify findings in reports. We assessed interrater reliability for DR and RI, and agreement between these two and with NLP., Results: For DR, interrater reliability was moderate for the presence of SBIs (k = 0.58, 95 % CI 0.46-0.69) and WMD (k = 0.49, 95 % CI 0.35-0.63), and moderate to substantial for characteristics of SBI and WMD. Agreement between DR and RI was substantial for the presence of SBIs and WMD, and fair to substantial for characteristics of SBIs and WMD. Agreement between NLP and DR was substantial for the presence of SBIs (k = 0.64, 95 % CI 0.53-0.76) and moderate (k = 0.52, 95 % CI 0.39-0.65) for the presence of WMD., Conclusions: Neuroimaging reports in routine care capture the presence of SBIs and WMD. An NLP can identify these findings (comparable to direct imaging review) and can likely be used for cohort identification.

Background and Purpose: Ischemic stroke patients with large vessel occlusion (LVO) could benefit from direct transportation to an intervention center for endovascular treatment, but non-LVO patients need rapid IV thrombolysis in the nearest center. Our aim was to evaluate prehospital triage strategies for suspected stroke patients in the United States., Methods: We used a decision tree model and geographic information system to estimate outcome of suspected stroke patients transported by ambulance within 4.5 hours after symptom onset. We compared the following strategies: (1) Always to nearest center, (2) American Heart Association algorithm (ie, directly to intervention center if a prehospital stroke scale suggests LVO and total driving time from scene to intervention center is <30 minutes, provided that the delay would not exclude from thrombolysis), (3) modified algorithms with a maximum additional driving time to the intervention center of <30 minutes, <60 minutes, or without time limit, and (4) always to intervention center. Primary outcome was the annual number of good outcomes, defined as modified Rankin Scale score of 0-2. The preferred strategy was the one that resulted in the best outcomes with an incremental number needed to transport to intervention center (NNTI) <100 to prevent one death or severe disability (modified Rankin Scale score of >2)., Results: Nationwide implementation of the American Heart Association algorithm increased the number of good outcomes by 594 (+1.0%) compared with transportation to the nearest center. The associated number of non-LVO patients transported to the intervention center was 16 714 (NNTI 28). The modified algorithms yielded an increase of 1013 (+1.8%) to 1369 (+2.4%) good outcomes, with a NNTI varying between 28 and 32. The algorithm without time limit was preferred in the majority of states (n=32 [65%]), followed by the algorithm with <60 minutes delay (n=10 [20%]). Tailoring policies at county-level slightly reduced the total number of transportations to the intervention center (NNTI 31)., Conclusions: Prehospital triage strategies can greatly improve outcomes of the ischemic stroke population in the United States, but increase the number of non-LVO stroke patients transported to an intervention center. The current American Heart Association algorithm is suboptimal as a nationwide policy and should be modified to allow more delay when directly transporting LVO-suspected patients to an intervention center.

Background: Randomised controlled trials are considered the gold standard for testing the efficacy of novel therapeutic interventions, and typically report the average treatment effect as a summary result. As the result of treatment can vary between patients, basing treatment decisions for individual patients on the overall average treatment effect could be suboptimal. We aimed to develop an individualised decision making tool to select an optimal revascularisation strategy in patients with complex coronary artery disease., Methods: The SYNTAX Extended Survival (SYNTAXES) study is an investigator-driven extension follow-up of a multicentre, randomised controlled trial done in 85 hospitals across 18 North American and European countries between March, 2005, and April, 2007. Patients with de-novo three-vessel and left main coronary artery disease were randomly assigned (1:1) to either the percutaneous coronary intervention (PCI) group or coronary artery bypass grafting (CABG) group. The SYNTAXES study ascertained 10-year all-cause deaths. We used Cox regression to develop a clinical prognostic index for predicting death over a 10-year period, which was combined, in a second stage, with assigned treatment (PCI or CABG) and two prespecified effect-modifiers, which were selected on the basis of previous evidence: disease type (three-vessel disease or left main coronary artery disease) and anatomical SYNTAX score. We used similar techniques to develop a model to predict the 5-year risk of major adverse cardiovascular events (defined as a composite of all-cause death, non-fatal stroke, or non-fatal myocardial infarction) in patients receiving PCI or CABG. We then assessed the ability of these models to predict the risk of death or a major adverse cardiovascular event, and their differences (ie, the estimated benefit of CABG versus PCI by calculating the absolute risk difference between the two strategies) by cross-validation with the SYNTAX trial (n=1800 participants) and external validation in the pooled population (n=3380 participants) of the FREEDOM, BEST, and PRECOMBAT trials. The concordance (C)-index was used to measure discriminative ability, and calibration plots were used to assess the degree of agreement between predictions and observations., Findings: At cross-validation, the newly developed SYNTAX score II, termed SYNTAX score II 2020, showed a helpful discriminative ability in both treatment groups for predicting 10-year all-cause deaths (C-index=0·73 [95% CI 0·69-0·76] for PCI and 0·73 [0·69-0·76] for CABG) and 5-year major adverse cardiovascular events (C-index=0·65 [0·61-0·69] for PCI and C-index=0·71 [0·67-0·75] for CABG). At external validation, the SYNTAX score II 2020 showed helpful discrimination (C-index=0·67 [0·63-0·70] for PCI and C-index=0·62 [0·58-0·66] for CABG) and good calibration for predicting 5-year major adverse cardiovascular events. The estimated treatment benefit of CABG over PCI varied substantially among patients in the trial population, and the benefit predictions were well calibrated., Interpretation: The SYNTAX score II 2020 for predicting 10-year deaths and 5-year major adverse cardiovascular events can help to identify individuals who will benefit from either CABG or PCI, thereby supporting heart teams, patients, and their families to select optimal revascularisation strategies., Funding: The German Heart Research Foundation and the Patient-Centered Outcomes Research Institute., (Copyright © 2020 Elsevier Ltd. All rights reserved.)

Background: Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial., Methods: We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel., Results: The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers)., Conclusions: Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

Background and Purpose: In patients with cryptogenic stroke and patent foramen ovale (PFO), the Risk of Paradoxical Embolism (RoPE) Score has been proposed as a method to estimate a patient-specific "PFO-attributable fraction"-the probability that a documented PFO is causally-related to the stroke, rather than an incidental finding. The objective of this research is to examine the relationship between this RoPE-estimated PFO-attributable fraction and the effect of closure in 3 randomized trials., Methods: We pooled data from the CLOSURE-I (Evaluation of the STARFlex Septal Closure System in Patients With a Stroke and/or Transient Ischemic Attack due to Presumed Paradoxical Embolism through a Patent Foramen Ovale), RESPECT (Randomized Evaluation of Recurrent Stroke Comparing PFO Closure to Established Current Standard of Care Treatment), and PC (Clinical Trial Comparing Percutaneous Closure of Patent Foramen Ovale [PFO] Using the Amplatzer PFO Occluder With Medical Treatment in Patients With Cryptogenic Embolism) trials. We examine the treatment effect of closure in high RoPE score (≥7) versus low RoPE score (<7) patients. We also estimated the relative risk reduction associated with PFO closure across each level of the RoPE score using Cox proportional hazard analysis. We estimated a patient-specific attributable fraction using a PC trial-compatible (9-point) RoPE equation (omitting the neuroradiology variable), as well as a 2-trial analysis using the original (10-point) RoPE equation. We examined the Pearson correlation between the estimated attributable fraction and the relative risk reduction across RoPE strata., Results: In the low RoPE score group (<7, n=912), the rate of recurrent strokes per 100 person-years was 1.37 in the device arm versus 1.68 in the medical arm (hazard ratio, 0.82 [0.42-1.59] P =0.56) compared with 0.30 versus 1.03 (hazard ratio, 0.31 [0.11-0.85] P =0.02) in the high RoPE score group (≥7, n=1221); treatment-by-RoPE score group interaction, P =0.12. The RoPE score estimated attributable fraction anticipated the relative risk reduction across all levels of the RoPE score, in both the 3-trial ( r =0.95, P <0.001) and 2-trial ( r =0.92, P <0.001) analyses., Conclusions: The RoPE score estimated attributable fraction is highly correlated to the relative risk reduction of device versus medical therapy. This observation suggests the RoPE score identifies patients with cryptogenic stroke who are likely to have a PFO that is pathogenic rather than incidental.

Background More than 500 000 sudden cardiac arrests (SCAs) occur annually in the United States. Clinical predictive models (CPMs) may be helpful tools to differentiate between patients who are likely to survive or have good neurologic recovery and those who are not. However, which CPMs are most reliable for discriminating between outcomes in SCA is not known. Methods and Results We performed a systematic review of the literature using the Tufts PACE (Predictive Analytics and Comparative Effectiveness) CPM Registry through February 1, 2020, and identified 81 unique CPMs of SCA and 62 subsequent external validation studies. Initial cardiac rhythm, age, and duration of cardiopulmonary resuscitation were the 3 most commonly used predictive variables. Only 33 of the 81 novel SCA CPMs (41%) were validated at least once. Of 81 novel SCA CPMs, 56 (69%) and 61 of 62 validation studies (98%) reported discrimination, with median c-statistics of 0.84 and 0.81, respectively. Calibration was reported in only 29 of 62 validation studies (41.9%). For those novel models that both reported discrimination and were validated (26 models), the median percentage change in discrimination was -1.6%. We identified 3 CPMs that had undergone at least 3 external validation studies: the out-of-hospital cardiac arrest score (9 validations; median c-statistic, 0.79), the cardiac arrest hospital prognosis score (6 validations; median c-statistic, 0.83), and the good outcome following attempted resuscitation score (6 validations; median c-statistic, 0.76). Conclusions Although only a small number of SCA CPMs have been rigorously validated, the ones that have been demonstrate good discrimination.

The machine learning community has become alert to the ways that predictive algorithms can inadvertently introduce unfairness in decision-making. Herein, we discuss how concepts of algorithmic fairness might apply in healthcare, where predictive algorithms are being increasingly used to support decision-making. Central to our discussion is the distinction between algorithmic fairness and algorithmic bias. Fairness concerns apply specifically when algorithms are used to support polar decisions (i.e., where one pole of prediction leads to decisions that are generally more desired than the other), such as when predictions are used to allocate scarce health care resources to a group of patients that could all benefit. We review different fairness criteria and demonstrate their mutual incompatibility. Even when models are used to balance benefits-harms to make optimal decisions for individuals (i.e., for non-polar decisions)-and fairness concerns are not germane-model, data or sampling issues can lead to biased predictions that support decisions that are differentially harmful/beneficial across groups. We review these potential sources of bias, and also discuss ways to diagnose and remedy algorithmic bias. We note that remedies for algorithmic fairness may be more problematic, since we lack agreed upon definitions of fairness. Finally, we propose a provisional framework for the evaluation of clinical prediction models offered for further elaboration and refinement. Given the proliferation of prediction models used to guide clinical decisions, developing consensus for how these concerns can be addressed should be prioritized., Competing Interests: Competing interestsThe authors declare no competing interests., (© The Author(s) 2020.)

Emerging data science techniques of predictive analytics expand the quality and quantity of complex data relevant to human health and provide opportunities for understanding and control of conditions such as heart, lung, blood, and sleep disorders. To realize these opportunities, the information sources, the data science tools that use the information, and the application of resulting analytics to health and health care issues will require implementation research methods to define benefits, harms, reach, and sustainability; and to understand related resource utilization implications to inform policymakers. This JACC State-of-the-Art Review is based on a workshop convened by the National Heart, Lung, and Blood Institute to explore predictive analytics in the context of implementation science. It highlights precision medicine and precision public health as complementary and compelling applications of predictive analytics, and addresses future research and training endeavors that might further foster the application of predictive analytics in clinical medicine and public health., (Copyright © 2020 American College of Cardiology Foundation. All rights reserved.)

Importance: Recent epidemiologic and therapeutic advances have transformed understanding of the role of and therapeutic approach to patent foramen ovale (PFO) in ischemic stroke. Patent foramen ovale is likely responsible for approximately 5% of all ischemic strokes and 10% of those occurring in young and middle-aged adults., Observations: Randomized clinical trials have demonstrated that, to prevent recurrent ischemic stroke in patients with PFO and an otherwise-cryptogenic index ischemic stroke, PFO closure is superior to antiplatelet medical therapy alone; these trials have provided some evidence that, among medical therapy options, anticoagulants may be more effective than antiplatelet agents., Conclusions and Relevance: These new data indicate a need to update classification schemes of causative mechanisms in stroke, developed in an era in which an association between PFO and stroke was viewed as uncertain. We propose a revised general nomenclature and classification framework for PFO-associated stroke and detailed revisions for the 3 major stroke subtyping algorithms in wide use.

Background and Purpose: To identify factors associated with prior stroke at presentation in patients with cryptogenic stroke (CS) and patent foramen ovale (PFO). Methods: We studied cross-sectional data from the International PFO Consortium Study (NCT00859885). Patients with first-ever stroke and those with prior stroke at baseline were analyzed for an association with PFO-related (right-to-left shunt at rest, atrial septal aneurysm, deep venous thrombosis, pulmonary embolism, and Valsalva maneuver) and PFO-unrelated factors (age, gender, BMI, hypertension, diabetes mellitus, hypercholesterolemia, smoking, migraine, coronary artery disease, aortic plaque). A multivariable analysis was used to adjust effect estimation for confounding, e.g., owing to the age-dependent definition of study groups in this cross-sectional study design. Results: We identified 635 patients with first-ever and 53 patients with prior stroke. Age, BMI, hypertension, diabetes mellitus, hypercholesterolemia, coronary artery disease, and right-to-left shunt (RLS) at rest were significantly associated with prior stroke. Using a pre-specified multivariable logistic regression model, age (Odds Ratio 1.06), BMI (OR 1.06), hypercholesterolemia (OR 1.90) and RLS at rest (OR 1.88) were strongly associated with prior stroke.Based on these factors, we developed a nomogram to illustrate the strength of the relation of individual factors to prior stroke. Conclusion: In patients with CS and PFO, the likelihood of prior stroke is associated with both, PFO-related and PFO-unrelated factors., (Copyright © 2020 Kahles, Michel, Hapfelmeier, Eberli, Zedde, Thijs, Kraemer, Engelter, Serena, Weimar, Mallmann, Luft, Hemelsoet, Thaler, Müller-Eichelberg, De Pauw, Sztajzel, Armon, Kent, Meier, Mattle, Fischer, Arnold, Mono and Nedeltchev.)

Aim: Despite broad interest in advancing personalized medicine, most evidence is currently derived from average results of clinical trials that may obscure heterogeneity of trial participants. Little is known currently about how patients view heterogeneity in trials and whether they can participate in methodological discussions about this concept. Materials & methods: In structured discussions with three focus groups involving 22 participants, we assessed how representatives of patient communities have used research to guide individual treatment decisions. Discussion themes were organized into a framework describing patient decision-making in four steps: decisions patients make in the course of care; information used to make decisions; sources for information; and quality of information. Results/conclusion: Patients prioritize information that reflects their own characteristics, preferences and values. They struggle applying clinical research to their own case.