Search

Specialty drugs are high-cost medications often used to treat complex chronic conditions. Even with insurance coverage, patients may face very high out-of-pocket costs, which in turn may restrict access. While the Inflation Reduction Act of 2022 included policies designed to reduce specialty drug costs, relatively few policies have been enacted during the past decade. In 2022-2023, we conducted a scoping literature review to identify a range of policy options and selected a set of 9 that have been regularly discussed or recently considered to present to an expert stakeholder panel to seek consensus on (1) the feasibility of implementing each policy and (2) its likely impact on drug costs. Experts rated only 1 policy highly on both feasibility and impact: grouping originator biologics and biosimilars under the same Medicare Part B reimbursement code. They rated 3 policies focused on setting payment limits as likely to have positive (downward) impact on costs but of uncertain feasibility. They considered 4 policies as uncertain on both criteria. Experts rated capping monthly out-of-pocket costs as feasible but unlikely to reduce specialty drug costs. Based on these results, we offer 4 recommendations to policymakers considering ways to reduce specialty drug costs., Competing Interests: None of the authors have any relevant conflicts of interest. D.K. is the creator of ExpertLens which was used to collect data for this study. Please see ICMJE form(s) for author conflicts of interest. These have been provided as supplementary materials., (© The Author(s) 2024. Published by Oxford University Press on behalf of Project HOPE - The People-To-People Health Foundation, Inc.)

Background: The transition from pediatric to adult care poses a significant health system-level challenge impeding the delivery of quality health services for youth with chronic health conditions. In Canada and globally, the transition to adult care is regarded as a top priority in adolescent health in need of readily applicable, adaptable, and relevant national metrics to evaluate and benchmark transition success across disease populations and clinical care settings. Unfortunately, existing literature fails to account for the lack of engagement from youth and caregivers in developing indicators, and its applicability across chronic conditions, primary care involvement, and health equity considerations., Objective: Our proposed study aims to establish a consensus-driven set of quality indicators for the transition to adult care that are universally applicable across physical, developmental, and mental health conditions, clinical care settings, and health jurisdictions., Methods: Using an integrated knowledge translation (iKT) approach, a panel comprising youth, caregivers, interdisciplinary health care providers, and health system leaders will be established to collaborate with our research team to ensure that the study methodology, materials, and knowledge dissemination are suitable and reflect the perspectives of youth and their families. We will then conduct an iterative 3-round Online Modified Delphi (OMD) study (n=160) to (1) compare and contrast the perspectives of youth, caregivers, health care providers, and health system leaders on quality indicators for transition; and (2) prioritize a key set of quality indicators for transition applicable across disease populations that are the most important, useful, and feasible in the Canadian context. Using the RAND/UCLA Appropriateness Method (RAM) multistage analytic approach, data from each panel and stakeholder group will be examined separately and compared to establish a key set of indicators endorsed by both panels., Results: The study is funded by the Canadian Institutes of Health Research and Physicians Services Incorporated., Conclusions: This study will produce quality indicators to evaluate and inform action equitably to improve transition from pediatric to adult care for youth and their families in Canada., International Registered Report Identifier (irrid): PRR1-10.2196/60860., (©Alene Toulany, Dmitry Khodyakov, Sarah Mooney, Lisa Stromquist, Katherine Bailey, Claire EH Barber, Michelle Batthish, Kristin Cleverley, Gina Dimitropoulos, Jan Willem Gorter, Danijela Grahovac, Ruth Grimes, Beverly Guttman, Michèle L Hébert, Tomisin John, Lisha Lo, Dorothy Luong, Laura MacGregor, Geetha Mukerji, Jacklynn Pidduck, Vjura Senthilnathan, Rayzel Shulman, Patricia Trbovich, Sarah EP Munce. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 10.09.2024.)

Patients with suspected or confirmed high-consequence infectious diseases (HCIDs) may present for care at any acute care facility. However, there are limited data to inform recommendations for the design, maintenance, and operation of isolation spaces for patients with suspected or confirmed HCIDs. To address this gap, we developed consensus statements by convening a group of 29 subject matter experts to participate in a modified Delphi process facilitated by a validated tool (the RAND-developed ExpertLens system). The subject matter experts participated in 3 consensus rounds, providing feedback and rating the appropriateness of 36 draft consensus statements. These draft statements were then revised based on their feedback. As a result, we developed 36 consensus statements addressing 5 domains: (1) patient room physical space, (2) doors and windows, (3) air handling, (4) electrical and plumbing, and (5) soiled utility rooms and waste management. These statements could inform the approaches of frontline acute care facilities when building new spaces or modifying existing spaces to enable appropriate HCID patient isolation and care.

Background: The current Institute of Medicine (IOM) pregnancy weight gain guidelines were developed using the best available evidence but were limited by substantial knowledge gaps. Some have raised concern that the guidelines for individuals affected by overweight or obesity are too high and contribute to short- and long-term complications for the mother and child., Objectives: To determine the association between pregnancy weight gain below the lower limit of the current IOM recommendations and risk of 10 adverse maternal and child health outcomes among individuals with overweight and obesity., Methods: We used data from a prospective cohort study of United States nulliparae with prepregnancy overweight (n = 955) or obesity (n = 897) followed from the first trimester to 2-7 y postpartum. We used multivariable Poisson regression to relate pregnancy weight gain z-scores with a severity-weighted composite outcome consisting of ≥1 of 10 adverse outcomes (gestational diabetes, preeclampsia, unplanned cesarean delivery, maternal postpartum weight increase >10 kg, maternal postpartum metabolic syndrome, infant death, stillbirth, preterm birth, small-for-gestational age birth, and childhood obesity)., Results: Pregnancy weight gain z-scores below, within, and above the IOM-recommended ranges occurred in 5%, 13%, and 80% of pregnancies with overweight and 17%, 13%, and 70% of pregnancies with obesity. There was a positive association between pregnancy weight gain z-scores and all adverse maternal outcomes, childhood obesity, and the composite outcome. Pregnancy weight gain z-scores below the lower limit of the recommended ranges (<6.8 kg for overweight, <5 kg for obesity) were not associated with the severity-weighted composite outcome. For example, compared with the lower limit, adjusted rate ratios (95% confidence interval) for z-scores of -2 standard deviations in pregnancies with overweight (equivalent to 3.6 kg at 40 wk) and obesity (-2.8 kg at 40 wk) were 0.99 (95% confidence interval [CI]: 0.91, 1.06) and 0.97 (95% CI: 0.87, 1.07)., Conclusions: These findings support arguments to decrease the lower limit of recommended weight gain ranges in these prepregnancy body mass index groups., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Introduction: Patient engagement and integrated knowledge translation (iKT) processes improve health outcomes and care experiences through meaningful partnerships in consensus-building initiatives and research. Consensus-building is essential for engaging a diverse group of experienced knowledge users in co-developing and supporting a solution where none readily exists or is less optimal. Patients and caregivers provide invaluable insights for building consensus in decision-making around healthcare, policy and research. However, despite emerging evidence, patient engagement remains sparse within consensus-building initiatives. Specifically, our research has identified a lack of opportunity for youth living with chronic health conditions and their caregivers to participate in developing consensus on indicators/benchmarks for transition into adult care. To bridge this gap and inform our consensus-building approach with youth/caregivers, this scoping review will synthesise the extent of the literature on patient and other knowledge user engagement in consensus-building healthcare initiatives., Methods and Analysis: Following the scoping review methodology from Joanna Briggs Institute, published literature will be searched in MEDLINE, EMBASE, CINAHL and PsycINFO databases from inception to July 2023. Grey literature will be hand-searched. Two independent reviewers will determine the eligibility of articles in a two-stage process, with disagreements resolved by a third reviewer. Included studies must be consensus-building studies within the healthcare context that involve patient engagement strategies. Data from eligible studies will be extracted and charted on a standardised form. Abstracted data will be analysed quantitatively and descriptively, according to specific consensus methodologies, and patient engagement models and/or strategies., Ethics and Dissemination: Ethics approval is not required for this scoping review protocol. The review process and findings will be shared with and informed by relevant knowledge users. Dissemination of findings will also include peer-reviewed publications and conference presentations. The results will offer new insights for supporting patient engagement in consensus-building healthcare initiatives., Protocol Registration: https://osf.io/beqjr., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Physician treatment preference may influence how risks are communicated in prostate cancer consultations. We identified persuasive language used when describing cancer prognosis, life expectancy, and side effects in relation to a physician's recommendation for aggressive (surgery/radiation) or nonaggressive (active surveillance/watchful waiting) treatment., Methods: A qualitative analysis was performed on transcribed treatment consultations of 40 men with low- and intermediate-risk prostate cancer across 10 multidisciplinary providers. Quotes pertaining to cancer prognosis, life expectancy, and side effects were randomized. Coders predicted physician treatment recommendations from isolated blinded quotes. Testing characteristics of consensus predictions against the physician's treatment recommendation were reported. Coders then identified persuasive strategies favoring aggressive/nonaggressive treatment for each quote. Frequencies of persuasive strategies favoring aggressive/nonaggressive treatment were reported. Logistic regression quantified associations between persuasive strategies and physician treatment recommendations., Results: A total of 496 quotes about cancer prognosis ( n  = 127), life expectancy ( n  = 51), and side effects ( n  = 318) were identified. The accuracy of predicting treatment recommendation based on individual quotes containing persuasive language ( n  = 256/496, 52%) was 91%. When favoring aggressive treatment, persuasive language downplayed side effect risks and amplified cancer risk (recurrence, progression, or mortality). Significant predictors ( P  < 0.05) of aggressive treatment recommendation included favorable side effect interpretation, downplaying side effects, and long time horizon for cancer risk due to longevity. When favoring nonaggressive treatment, persuasive language amplified side effect risks and downplayed cancer risk. Significant predictors of nonaggressive treatment recommendation included unfavorable side effect interpretation, favorable interpretation of cancer risk, and short time horizon for cancer risk due to longevity., Conclusions: Physicians use persuasive language favoring their preferred treatment, regardless of whether their recommendation is appropriate., Implications: Clinicians should quantify risk so patients can judge potential harm without solely relying on persuasive language., Highlights: Physicians use persuasive language favoring their treatment recommendation when communicating risks of prostate cancer treatment, which may influence a patient's treatment choice.Coders predicted physician treatment recommendations based on isolated, randomized quotes about cancer prognosis, life expectancy, and side effects with 91% accuracy.Qualitative analysis revealed that when favoring nonaggressive treatment, physicians used persuasive language that amplified side effect risks and downplayed cancer risk. When favoring aggressive treatment, physicians did the opposite.Providers should be cognizant of using persuasive strategies and aim to provide quantified assessments of risk that are jointly interpreted with the patient so that patients can make evidence-based conclusions regarding risks without solely relying on persuasive language., Competing Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this study was provided entirely by a by the Career Development Award (K08 CA230155 to Timothy J. Daskivich) from the National Cancer Institute. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Background: Effective communication of treatment side effects (SE) is critical for shared decision-making (SDM) in localized prostate cancer. We sought to qualitatively characterize how physicians communicate SE in consultations., Methods: We transcribed 50 initial prostate cancer treatment consultations across nine multidisciplinary providers (Urologists, Radiation Oncologists, Medical Oncologists) at our tertiary referral, academic center. Coders identified quotes describing SE and used an inductive approach to establish a hierarchy for granularity of communication: (1) not mentioned, (2) name only, (3) generalization("high"), (4) average incidence without timepoint, (5) average incidence with timepoint, and (6) precision estimate. We reported the most granular mode of communication for each SE throughout the consultation overall and across specialty and tumor risk., Results: Among consultations discussing surgery (n = 40), erectile dysfunction (ED) and urinary incontinence (UI) were omitted in 15% and 12%, not quantified (name only or generalization) in 47% and 30%, and noted as average incidence without timeline in 8% and 8%, respectively. In only 30% and 49% were ED and UI quantified with timeline (average incidence with timeline or precision estimate), respectively. Among consultations discussing radiation (n = 36), irritative urinary symptoms, ED, and other post-radiotherapy SE were omitted in 22%, 42%, and 64-67%, not quantified in 61%, 33%, and 23-28%, and noted as average incidence without timeline in 8%, 22%, and 6-8%, respectively. In only 3-8% were post-radiotherapy SE quantified with timeline. Specialty concordance (but not tumor risk) was associated with higher granularity of communication, though physicians frequently failed to quantify specialty-concordant SE., Conclusions: SE was often omitted, not quantified, and/or lacked a timeline in treatment consultations in our sample. Physicians should articulate, quantify, and assign a timeline for SE to optimize SDM., (© 2024. The Author(s).)

Background: The Institute of Medicine pregnancy weight gain guidelines were developed without evidence linking high weight gain to maternal cardiometabolic disease and child obesity. The upper limit of current recommendations may be too high for the health of the pregnant individual and child., Objectives: The aim of this study was to identify the range of pregnancy weight gain for pregnancies within a normal body mass index (BMI) range that balances the risks of high and low weight gain by simultaneously considering 10 different health conditions., Methods: We used data from an United States prospective cohort study of nulliparae followed until 2 to 7 y postpartum (N = 2344 participants with a normal BMI). Pregnancy weight gain z-score was the main exposure. The outcome was a composite consisting of the occurrence of ≥1 of 10 adverse health conditions that were weighted for their seriousness. We used multivariable Poisson regression to relate weight gain z-scores with the weighted composite outcome., Results: The lowest risk of the composite outcome was at a pregnancy weight gain z-score of -0.6 SD (standard deviation) (equivalent to 13.1 kg at 40 wk). The weight gain ranges associated with no more than 5%, 10%, and 20% increase in risks were -1.0 to -0.2 SD (11.2-15.3 kg), -1.4 to 0 SD (9.4-16.4 kg), and -2.0 to 0.4 SD (7.0-18.9 kg). When we used a lower threshold to define postpartum weight increase in the composite outcome (>5 kg compared with >10 kg), the ranges were 1.6 to -0.7 SD (8.9-12.6 kg), -2.2 to -0.3 SD (6.3-14.7 kg), and ≤0.2 SD (≤17.6 kg). Compared with the ranges of the current weight gain guidelines (-0.9 to -0.1 SD, 11.5-16 kg), the lower limits from our data tended to be lower while upper limits were similar or lower., Conclusions: If replicated, our results suggest that policy makers should revisit the recommended pregnancy weight gain range for individuals within a normal BMI range., (Copyright © 2023 American Society for Nutrition. Published by Elsevier Inc. All rights reserved.)

Background: Clinicians treating cancer-related pain with opioids regularly encounter nonmedical stimulant use (i.e., methamphetamine, cocaine), yet there is little evidence-based management guidance. The aim of the study is to identify expert consensus on opioid management strategies for an individual with advanced cancer and cancer-related pain with nonmedical stimulant use according to prognosis., Methods: The authors conducted two modified Delphi panels with palliative care and addiction experts. In Panel A, the patient's prognosis was weeks to months and in Panel B the prognosis was months to years. Experts reviewed, rated, and commented on the case using a 9-point Likert scale from 1 (very inappropriate) to 9 (very appropriate) and explained their responses. The authors applied the three-step analytical approach outlined in the RAND/UCLA to determine consensus and level of clinical appropriateness of management strategies. To better conceptualize the quantitative results, they thematically analyzed and coded participant comments., Results: Consensus was achieved for all management strategies. The 120 Experts were mostly women (47 [62%]), White (94 [78%]), and physicians (115 [96%]). For a patient with cancer-related and nonmedical stimulant use, regardless of prognosis, it was deemed appropriate to continue opioids, increase monitoring, and avoid opioid tapering. Buprenorphine/naloxone transition was inappropriate for a patient with a short prognosis and of uncertain appropriateness for a patient with a longer prognosis., Conclusion: Study findings provide urgently needed consensus-based guidance for clinicians managing cancer-related pain in the context of stimulant use and highlight a critical need to develop management strategies to address stimulant use disorder in people with cancer., Plain Language Summary: Among palliative care and addiction experts, regardless of prognosis, it was deemed appropriate to continue opioids, increase monitoring, and avoid opioid tapering in the context of cancer-related pain and nonmedical stimulant use. Buprenorphine/naloxone transition as a harm reduction measure was inappropriate for a patient with a short prognosis and of uncertain appropriateness for a patient with a longer prognosis., (© 2023 American Cancer Society.)

The Delphi method is an iterative, anonymous, group-based process for eliciting and aggregating opinion on a topic to explore the existence of consensus among experts. The year 2023 marks the 60th anniversary of the first peer-reviewed journal article on the Delphi method. Originally developed for operations research, this method is now applied extensively by researchers representing diverse scientific fields. We used a bibliometric analysis to describe general trends in the expansion of its use across disciplines over time. We conducted a systematic literature search for all English-language, peer-reviewed journal articles on the Delphi method through its first 60 years. We found 19,831 articles: 96.8% (n = 19,204) on the actual use of the Delphi method in an empirical study and 3.2% (n = 627) describing, examining, or providing some guidance on how to use the Delphi method. Almost half (49.9%) of all articles were published in the 2010s and an additional third (32.5%) in the first few years of the 2020s. Nearly two-thirds (65%, n = 12,883) of all published articles have appeared in medical journals, compared to 15% in science and technology (n = 3,053) or social science (n = 3,016) journals. We conclude that the expanded use of the Delphi method has been driven largely by the medical field, though social scientists and technologists continue to be at the forefront of methodological work on the Delphi method. Therefore, we call for greater transdisciplinary collaboration on methodological guidance and standards for the Delphi method., Competing Interests: DK and SG are members of the ExpertLens team. ExpertLens is a RAND-developed platform for conducting online Delphi panels. DK consults on the Delphi method at Pfizer. S.G.’s spouse is a salaried employee of Eli Lilly and Company and owns stock. S.G. has accompanied his spouse on company-sponsored travel. All other authors declare that they have no conflicts of interest. All other co-authors report no competing interests. This does not alter our adherence to PLOS ONE policies on sharing data and materials., (Copyright: © 2023 Khodyakov et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Importance: Opioid misuse and opioid use disorder (OUD) are important comorbidities in people with advanced cancer and cancer-related pain, but there is a lack of consensus on treatment., Objective: To develop consensus among palliative care and addiction specialists on the appropriateness of various opioid management strategies in individuals with advanced cancer-related pain and opioid misuse or OUD., Design, Setting, and Participants: For this qualitative study, using ExpertLens, an online platform and methodology for conducting modified Delphi panels, between August and October 2020, we conducted 2 modified Delphi panels to understand the perspectives of palliative and addiction clinicians on 3 common clinical scenarios varying by prognosis (weeks to months vs months to years). Of the 129 invited palliative or addiction medicine specialists, 120 participated in at least 1 round. A total of 84 participated in all 3 rounds., Main Outcomes and Measures: Consensus was investigated for 3 clinical scenarios: (1) a patient with a history of an untreated opioid use disorder, (2) a patient taking more opioid than prescribed, and (3) a patient using nonprescribed benzodiazepines., Results: Participants were mostly women (47 [62%]), White (94 (78 [65%]), and held MD/DO degrees (115 [96%]). For a patient with untreated OUD, regardless of prognosis, it was deemed appropriate to begin treatment with buprenorphine/naloxone and inappropriate to refer to a methadone clinic. Beginning split-dose methadone was deemed appropriate for patients with shorter prognoses and of uncertain appropriateness for those with longer prognoses. Beginning a full opioid agonist was deemed of uncertain appropriateness for those with a short prognosis and inappropriate for those with a longer prognosis. Regardless of prognosis, for a patient with no medical history of OUD taking more opioids than prescribed, it was deemed appropriate to increase monitoring, inappropriate to taper opioids, and of uncertain appropriateness to increase the patient's opioids or transition to buprenorphine/naloxone. For a patient with a urine drug test positive for non-prescribed benzodiazepines, regardless of prognosis, it was deemed appropriate to increase monitoring, inappropriate to taper opioids and prescribe buprenorphine/naloxone., Conclusions and Relevance: The findings of this qualitative study provide urgently needed consensus-based guidance for clinicians and highlight critical research and policy gaps.

Purpose: Men with prostate cancer prefer patient-specific, quantitative assessments of longevity in shared decision making. We sought to characterize how physicians communicate the 3 components of competing risks-life expectancy (LE), cancer prognosis and treatment-related survival benefit-in treatment consultations., Materials and Methods: Conversation related to LE, cancer prognosis and treatment-related survival benefit was identified in transcripts from treatment consultations of 42 men with low- and intermediate-risk disease across 10 multidisciplinary providers. Consensus of qualitative coding by multiple reviewers noted the most detailed mode of communication used to describe each throughout the consultation., Results: Physicians frequently failed to provide patient-specific, quantitative estimates of LE and cancer mortality. LE was omitted in 17% of consultations, expressed as a generalization (eg "long"/"short") in 17%, rough number of years in 31%, probability of mortality/survival at an arbitrary timepoint in 17% and in only 19% as a specific number of years. Cancer mortality was omitted in 24% of consultations, expressed as a generalization in 7%, years of expected life in 2%, probability at no/arbitrary timepoint in 40% and in only 26% as the probability at LE. Treatment-related survival benefit was often omitted; cancer mortality was reported without treatment in 38%, with treatment in 10% and in only 29% both with and without treatment. Physicians achieved "trifecta"-1) quantifying probability of cancer mortality 2) with and without treatment 3) at the patient's LE-in only 14% of consultations., Conclusions: Physicians often fail to adequately quantify competing risks. We recommend the "trifecta" approach, reporting 1) probability of cancer mortality 2) with and without treatment 3) at the patient's LE.

Objective: The authors sought to identify the most promising strategies for improving the mental health guardianship process in Los Angeles County for adults with mental illness who are gravely disabled., Methods: In May and June 2019, 56 experts, working in hospitals or outpatient facilities or representing legal, advocacy, policy, or forensic organizations, participated in an online modified-Delphi panel, rating the ethical appropriateness, impact on care quality, efficiency, and feasibility of nine strategies for improvement of mental health guardianship. Agreement was determined with the RAND/UCLA appropriateness method, and comments were thematically analyzed., Results: The strategy ranked highest by the participating experts was improving the administrative functioning and judicial processes of entities involved in mental health guardianship proceedings-it was the only strategy that achieved agreement among panelists and was rated highly on all four criteria. Other preferred strategies were enhancing the ability of assertive outpatient mental health teams to serve individuals before they experience a crisis and expanding the continuum of unlocked residential treatment settings., Conclusions: Opportunities exist to improve all stages of the mental health guardianship process. Experts favored strategies that streamline administrative processes, facilitate community integration into treatment, and ensure fidelity to best practices. Improving the mental health guardianship process has the potential to speed up delivery of services, better manage resources, and increase access to treatment for individuals with mental illness who are gravely disabled.

Introduction: Multistakeholder engagement is crucial for conducting health services research. Delphi-based methodologies combining iterative rounds of questions with feedback on and discussion of group results are a well-documented approach to multistakeholder engagement. This study develops hypotheses about the impact of panel composition and topic on the propensity and meaningfulness of response changes in multistakeholder modified-Delphi panels., Methods: We conducted three online modified-Delphi (OMD) multistakeholder panels using the same protocol. We assigned 60 maternal and child health professionals to a homogeneous (professionals only) panel, 60 pregnant or postpartum women (patients) to a homogeneous panel, and 30 professionals and 30 patients to a mixed panel. In Round 1, participants rated the seriousness of 11 maternal and child health outcomes using a 0-100 scale and explained their ratings. In Round 2, participants saw their own and their panel's Round 1 results and discussed them using asynchronous, anonymous discussion boards moderated by the study investigators. In Round 3, participants revised their original ratings. Our outcome measures included binary indicators of response changes to ratings of the low, medium and high severity maternal and child health outcomes and their meaningfulness, measured by a change of 10 or more points., Results: Participants changed 818 of 1491 (55%) of responses; the majority of response changes were meaningful. Patterns of response changes were different for patients and professionals and for different levels of outcome seriousness. Using study results and the literature, we developed three hypotheses. First, OMD participants, regardless of their stakeholder group, are more likely to change their responses on preference-sensitive topics where there is a range of viable alternatives or perspectives. Second, patients are more likely to change their responses and to do so meaningfully in mixed panels, whereas professionals are more likely to do so in homogeneous panels. Third, the association between panel composition and response change varies according to the topic (e.g., the level of outcome seriousness)., Conclusions: Results of our work not only helped generate empirically derived hypotheses to be tested in future research but also offer practical recommendations for designing multistakeholder OMD panels., Patient or Public Contribution: Pregnant or postpartum women were involved in this study., (© 2021 The Authors. Health Expectations published by John Wiley & Sons Ltd.)

Objective: To standardize the clinical definition of opioid withdrawal in neonates to address challenges in clinical care, quality improvement, research, and public policy for this patient population., Study Design: Between October and December 2020, we conducted 2 modified-Delphi panels using ExpertLens, a virtual platform for performing iterative expert engagement panels. Twenty clinical experts specializing in care for the substance-exposed mother-neonate dyad explored the necessity of key evidence-based clinical elements in defining opioid withdrawal in the neonate leading to a diagnosis of neonatal abstinence syndrome (NAS)/neonatal opioid withdrawal syndrome (NOWS). Expert consensus was assessed using descriptive statistics, the RAND/UCLA Appropriateness Method, and thematic analysis of participants' comments., Results: Expert panels concluded the following were required for diagnosis: in utero exposure (known by history, not necessarily by toxicology testing) to opioids with or without the presence of other psychotropic substances, and the presence of at least two of the most common clinical signs characteristic of withdrawal (excessive crying, fragmented sleep, tremors, increased muscle tone, gastrointestinal dysfunction)., Conclusions: Results indicate that both a known history of in utero opioid exposure and a distinct set of withdrawal signs are necessary to standardize a definition of neonatal withdrawal. Implementation of a standardized definition requires both patient engagement and a mother-neonate dyadic approach mindful of program and policy implications., (Published by Elsevier Inc.)

Competing Interests: Conflict-of-Interest Disclosures: Dr. Michener discloses serving as Principal Investigator for the Practical Playbook™ and as consultant for HHS, OAHS, NIH, CDC, HRSA, UC Davis, and Rockefeller University.

Importance: Metrics that detect low-value care in common forms of health care data, such as administrative claims or electronic health records, primarily focus on tests and procedures but not on medications, representing a major gap in the ability to systematically measure low-value prescribing., Objective: To develop a scalable and broadly applicable metric that contains a set of quality indicators (EVOLV-Rx) for use in health care data to detect and reduce low-value prescribing among older adults and that is informed by diverse stakeholders' perspectives., Design, Setting, and Participants: This qualitative study used an online modified-Delphi method to convene an expert panel of 15 physicians and pharmacists. This panel, comprising clinicians, health system leaders, and researchers, was tasked with rating and discussing candidate low-value prescribing practices that were derived from medication safety criteria; peer-reviewed literature; and qualitative studies of patient, caregiver, and physician perspectives. The RAND ExpertLens online platform was used to conduct the activities of the panel. The panelists were engaged for 3 rounds between January 1 and March 31, 2021., Main Outcomes and Measures: Panelists used a 9-point Likert scale to rate and then discuss the scientific validity and clinical usefulness of the criteria to detect low-value prescribing practices. Candidate low-value prescribing practices were rated as follows: 1 to 3, indicating low validity or usefulness; 3.5 to 6, uncertain validity or usefulness; and 6.5 to 9, high validity or usefulness. Agreement among panelists and the degree of scientific validity and clinical usefulness were assessed using the RAND/UCLA (University of California, Los Angeles) Appropriateness Method., Results: Of the 527 low-value prescribing recommendations identified, 27 discrete candidate low-value prescribing practices were considered for inclusion in EVOLV-Rx. After round 1, 18 candidate practices were rated by the panel as having high scientific validity and clinical usefulness (scores of ≥6.5). After round 2 panel deliberations, the criteria to detect 19 candidate practices were revised. After round 3, 18 candidate practices met the inclusion criteria, receiving final median scores of 6.5 or higher for both scientific validity and clinical usefulness. Of those practices that were not included in the final version of EVOLV-Rx, 3 received high scientific validity (scores ≥6.5) but uncertain clinical usefulness (scores <6.5) ratings, whereas 6 received uncertain scientific validity rating (scores <6.5)., Conclusions and Relevance: This study culminated in the development of EVOLV-Rx and involved a panel of experts who identified the 18 most salient low-value prescribing practices in the care of older adults. Applying EVOLV-Rx may enhance the detection of low-value prescribing practices, reduce polypharmacy, and enable older adults to receive high-value care across the full spectrum of health services.

Importance: Opioid use disorder (OUD) is an important comorbidity in individuals with advanced cancer, in whom pain is common. Full-agonist opioid medications are the cornerstone of cancer pain management, but the existing literature does not address how to manage cancer pain in patients with OUD., Objective: To conduct an expert panel to develop consensus on the appropriateness of management of cancer pain in individuals with co-occurring advanced cancer and OUD., Evidence Review: A 3-round modified Delphi process was completed from August to October 2020 with 2 cases: patient with advanced cancer, pain, and OUD treated with buprenorphine-naloxone or methadone. Participants rated management strategies in round 1, discussed results in round 2, and provided final responses in round 3. ExpertLens, an online approach to conducting modified Delphi panels, was used. Participants were experts in palliative care, addiction, or both, recruited by email from palliative care and addiction-focused professional groups, lists from prior studies, and snowball sampling. Data analysis was performed from November 2020 to July 2021., Findings: Of 120 experts (median age, 40-49 years), most were White (78 participants [94%]), female (74 participants [62%]), and held MD or DO degrees (115 participants [96%]); 84 (70%) participated in all rounds. For a patient with OUD taking buprenorphine-naloxone, it was deemed appropriate to continue buprenorphine-naloxone with thrice-daily dosing. Continuing buprenorphine-naloxone and adding a full-agonist opioid was deemed to be appropriate for patients with a prognosis of weeks to months and of uncertain appropriateness for patients with a prognosis of months to years. For a patient with OUD taking methadone dispensed at a methadone clinic, it was deemed appropriate to take over prescribing and dose twice or thrice daily. Continuing methadone daily while adding another full-agonist opioid was deemed appropriate for patients with a prognosis of weeks to months and of uncertain appropriateness for those with a prognosis of months to years., Conclusions and Relevance: The findings of this qualitative study provide urgently needed, consensus-based guidance for clinicians and highlight critical research and policy gaps needed to facilitate implementation.

Background: Neonatal abstinence syndrome (NAS) is a postnatal withdrawal syndrome that most commonly results from prenatal opioid exposure. Every 15 minutes, an infant is born in the United States with signs of NAS. The field lacks a standardized clinical definition of NAS, complicating discussions on programmatic and policy development to support opioid-exposed mothers and infants., Objective: The goal of this paper is to describe a protocol for a systematic expert panel process to inform the development of a clinical definition of NAS., Methods: We will conduct two three-round online modified-Delphi panels using the ExpertLens system and will follow the recommendations for Conducting and REporting of DElphi Studies (CREDES). One panel will focus on developing key components of a clinical definition of NAS, and the second panel will focus on neonatal opioid withdrawal syndrome (NOWS), which is a term that has come into use to differentiate opioid-exposed infants from infants exposed to other substances in utero. However, there is lack of agreement on the precise clinical definition of NOWS and how it is distinct from or overlaps with NAS. Each panel will complete two rating rounds and a discussion round using a similar protocol. We will analyze all rating data descriptively and determine the presence of agreement within and between the two panels. We will also perform thematic analysis of the qualitative comments to contextualize the panel findings., Results: The panels were convened between October 29 and December 17, 2020. Their results were disseminated and discussed at a national conference on NAS that took place on March 17-18, 2021., Conclusions: A standardized clinical definition of NAS will help to better characterize NAS incidence and to design effective clinical, public health, and policy interventions to support opioid-exposed mother-infant dyads., International Registered Report Identifier (irrid): DERR1-10.2196/25387., (©Dmitry Khodyakov, Shahla M Jilani, Stephanie Dellva, Laura J Faherty. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 07.09.2021.)

Executive Summary: The article examines whether subjective performance assessments from health system executives match objective performance assessments and qualitatively explores ways to achieve high performance. We interviewed 138 C-suite executives of 24 health systems in California, Minnesota, Washington, and Wisconsin between 2017 and 2019. We used maximum variation sampling to select health systems to achieve diversity in performance on objective measures of clinical performance. Our interviews focused on executives' perceptions of their own health system's performance and factors they thought generally contributed to high performance. In our analysis, we grouped health systems based on objective performance levels (high, medium, and low) used in sampling, compared objective performance ratings with executives' subjective performance assessments, and used thematic analysis to identify reasons for subjective assessment of health system performance and levers of high performance in general. There was poor agreement between objective and subjective performance assessments (kappa = 0.082). Subjective assessments were higher than objective assessments and captured more factors than are typically considered in performance accountability and value-based payment initiatives. Executives whose views were inconsistent with objective performance assessments did not cite clinical care quality per se as the basis for their assessment, focusing instead on market competition, financial performance, and high customer satisfaction and loyalty. Executives who cited clinical quality metrics as the basis of their assessment offered subjective ratings consistent with objective ratings. Executives identified organizational culture, organizational governance, and staff engagement as levers for achieving high performance. Future research should explore the benefits and drawbacks of considering subjective performance assessments in value-based payment initiatives., Competing Interests: The authors declare no conflicts of interest., (Copyright © 2021 Foundation of the American College of Healthcare Executives.)

Background: Current pregnancy weight gain guidelines were developed based on implicit assumptions of a small group of experts about the relative seriousness of adverse health outcomes. Therefore, they will not necessarily reflect the values of women., Objective: To estimate the seriousness of 11 maternal and child health outcomes that have been consistently associated with pregnancy weight gain by engaging patients and health professionals., Methods: We collected data using an online panel approach with a modified Delphi structure. We selected a purposeful sample of maternal and child health professionals (n = 84) and women who were pregnant or recently postpartum (patients) (n = 82) in the United States as panellists. We conducted three concurrent panels: professionals only, patients only, and patients and professionals. During a 3-round online modified Delphi process, participants rated the seriousness of health outcomes (Round 1), reviewed and discussed the initial results (Round 2), and revised their original ratings (Round 3). Panellists assigned seriousness ratings (0, [not serious] to 100 [most serious]) for infant death, stillbirth, preterm birth, gestational diabetes, preeclampsia, small-for-gestational-age (SGA) birth, large-for-gestational-age (LGA) birth, unplanned caesarean delivery, maternal obesity, childhood obesity, and maternal metabolic syndrome., Results: Each panel individually came to a consensus on all seriousness ratings. The final median seriousness ratings combined across all panels were highest for infant death (100), stillbirth (95), preterm birth (80), and preeclampsia (80). Obesity in children, metabolic syndrome in women, obesity in women, and gestational diabetes had median seriousness ratings ranging from 55 to 65. The lowest seriousness ratings were for SGA birth, LGA birth, and unplanned caesarean delivery (30-40)., Conclusion: Professionals and women rate some adverse outcomes as being more serious than others. These ratings can be used to establish the range of pregnancy weight gain associated with the lowest risk of a broad range of maternal and child health outcomes., (© 2020 John Wiley & Sons Ltd.)

Introduction: Management of opioid misuse and opioid use disorder (OUD) among individuals with serious illness is an important yet understudied issue. Palliative care clinicians caring for individuals with serious illness, many of whom may live for months or years, describe a complex tension between weighing the benefits of opioids, which are considered a cornerstone of pain management in serious illness, and serious opioid-related harms like opioid misuse and OUD. And yet, little literature exists to inform the management of opioid misuse and OUDs among individuals with serious illness. Our objective is to provide evidence-based management guidance to clinicians caring for individuals with serious illness who develop opioid misuse or OUD., Methods and Analysis: We chose a modified Delphi approach, which is appropriate when empirical evidence is lacking and expert input must be used to shape clinical guidance. We sought to recruit 60 clinicians with expertise in palliative care, addiction or both to participate in this study. We created seven patient cases that capture important management challenges in individuals with serious illness prescribed opioid therapy. We used ExpertLens, an online platform for conducting modified Delphi panels. Participants completed three rounds of data collection. In round 1, they rated and commented on the appropriateness of management choices for cases. In round 2, participants reviewed and discussed their own and other participants' round 1 numerical responses and comments. In round 3 (currently ongoing), participants again reviewed rounds 1 and 2, and are allowed to change their final numerical responses. We used ExpertLens to automatically identify whether there is consensus, or disagreement, among responses in panels. Only round 3 responses will be used to assess final consensus and disagreement., Ethics and Dissemination: This project received ethical approval from the University of Pittsburgh's Institutional Review Board (study 19110301) and the RAND Institutional Research Board (study 2020-0142). Guidance from this work will be disseminated through national stakeholder networks to gain buy-in and endorsement. This study will also form the basis of an implementation toolkit for clinicians caring for individuals with serious illness who are at risk of opioid misuse or OUD., Competing Interests: Competing interests: This grant was supported by Cambia Health Foundation (PI Dr Merlin)., (© Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Evidence to guide clinical decision making for pain management in nursing home residents is scant., Objective: Our objective was to explore the extent of consensus among expert stakeholders regarding what analgesic issues should be prioritized for comparative-effectiveness studies of beneficial and adverse effects of analgesic regimens in nursing home residents., Methods: Two stakeholder panels (nurses only and a mix of clinicians/researchers) were engaged (n = 83). During a three-round online modified Delphi process, participants rated and commented on the need for new evidence on nonopioid analgesic regimens and opioid regimens, short-term adverse effects, long-term adverse effects, comorbid conditions, and other factors in the nursing home setting (9-point scale; 1 = not essential to 9 = very essential to obtain new evidence). The quantitative data were analyzed to determine the existence of consensus using an approach from the RAND/UCLA Appropriateness Method User's Manual. The qualitative data, consisting of participant explanations of their numeric ratings, were thematically analyzed by an experienced qualitative researcher., Results: For nursing home residents, evidence generation was deemed essential for opioids, gabapentin (alone or with serotonin norepinephrine reuptake inhibitors [SNRIs]), and nonsteroid anti-inflammatory drugs with SNRIs. Experts prioritized the following outcomes as essential: long-term adverse effects, including delirium, cognitive decline, and decline in activities of daily living (ADLs). Kidney disease and depression were deemed essential conditions to consider in studies of pain medications. Coprescribing analgesic regimens with benzodiazepines, sedating medications, serotonergic medications, and non-SNRI antidepressants were considered essential areas of study. Experts noted that additional study was essential in residents with moderate/severe cognitive impairment and limitations in ADLs., Conclusions: Stakeholder priorities for more evidence reflect concerns related to treating medically complex residents with complex drug regimens and included long-term adverse effects, coprescribing, and sedating medications. Carefully conducted observational studies are needed to address the vast evidence gap for nursing home residents.

Objectives: To describe the extent and nature of changes in participants' responses after exposure to group feedback and discussion in modified-Delphi panels and to identify factors affecting those changes., Methods: We analyzed data from 2 online modified-Delphi panels, each consisting of 2 rating rounds and an online discussion round. We included responses from 55 participants who answered 38 questions in both rating rounds. Because not all participants answered each question twice, our sample consisted of 1846 cases (response changes). We used mixed-effect logistic and multinomial logistic regression to identify factors predicting response changes and their direction relative to group median-our consensus measure., Results: Participants changed, on average, 49% of their responses. A response was changed in 47% of the 1846 cases: 28% of responses were changed toward consensus and 19% away from it. Although some measures of subjective participation experiences had a marginally significant impact on the propensity and direction of response changes, several objective measures of discussion engagement were statistically significant predictors of both the presence and direction of response changes., Conclusion: Our results illustrate the nature of response changes and highlight the importance of exposing participants to alternative perspectives and encouraging them to explain their perspectives., (Copyright © 2020 ISPOR–The Professional Society for Health Economics and Outcomes Research. Published by Elsevier Inc. All rights reserved.)

Background: Despite significant investments in health information technology (IT), the technology has not yielded the intended performance effects or transformational change. We describe activities that health systems are pursuing to better leverage health IT to improve performance., Methods: We conducted semi-structured telephone interviews with C-suite executives from 24 U.S. health systems in four states during 2017-2019 and analyzed the data using a qualitative thematic approach., Results: Health systems reported two broad categories of activities: laying the foundation to improve performance with IT and using IT to improve performance. Within these categories, health systems were engaged in similar activities but varied greatly in their progress. The most substantial effort was devoted to the first category, which enabled rather than directly improved performance, and included consolidating to a single electronic health record (EHR) platform and common data across the health system, standardizing data elements, and standardizing care processes before using the EHR to implement them. Only after accomplishing such foundational activities were health systems able to focus on using the technology to improve performance through activities such as using data and analytics to monitor and provide feedback, improving uptake of evidence-based medicine, addressing variation and overuse, improving system-wide prevention and population health management, and making care more convenient., Conclusions and Implications: Leveraging IT to improve performance requires significant and sustained effort by health systems, in addition to significant investments in hardware and software. To accelerate change, better mechanisms for creating and disseminating best practices and providing advanced technical assistance are needed., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Background: Bleeding disorders (BD) are under-recognized in adolescents with heavy menstrual bleeding (HMB)., Objectives: The lack of clinical guidelines and variable symptomatic management of HMB created the imperative to standardize HMB care to identify and manage BD in adolescents., Methods: We convened an international working group (WG), utilized the results of a literature review to define knowledge gaps in HMB care, and used the collective clinical experience of the WG to develop care considerations for adolescents with BD and HMB. We then solicited input on the appropriateness of HMB care considerations from expert stakeholders representing hematology, adolescent medicine, and obstetrics-gynecology. We conducted an expert panel online, using the ExpertLens platform. During a three-round online modified-Delphi process, the expert panel rated the appropriateness of 21 care considerations using a 9-point scale to designate care as appropriate (7-9), uncertain (4-6), or inappropriate (1-3) covering screening for BD, the laboratory work-up, and management of adolescents with BD that present with HMB. We used the RAND/UCLA appropriateness method to determine the existence of consensus among the interdisciplinary panel of experts., Results: Thirty-nine experts participated in the panel. The experts rated fifteen HMB care considerations as appropriate, six as uncertain, and none as inappropriate., Conclusions: The HMB care statements represent the first set of HMB care considerations in adolescents with BD, developed with broad expert input on appropriateness. Although likely to be of interest to a range of clinicians who routinely manage adolescents with HMB, additional research is required in many key areas., (© 2020 International Society on Thrombosis and Haemostasis.)

Introduction: Complex population health interventions that are effective in one context may not be effective elsewhere, and may even be harmful. As such, an intervention may require adaptation to ensure it fits with a new context. To date, there is no overarching guidance to help researchers to adapt and evaluate interventions in new contexts, and no criteria to support research funders or journals assess proposed or reported adaptations or evaluation. There is limited assistance for policy-makers and practitioners to decide if evidence-informed interventions are appropriate to their context, or if adaptation and further evaluation is needed. This Delphi exercise will contribute to the development of guidance for these communities to support the adaptation, implementation and/or re-evaluation of complex population health interventions in new contexts., Methods: We will conduct a Delphi consensus exercise to gather expert opinion from researchers, research funders, journal editors and policy-makers. Expert opinion will be sought on: appropriate definitions and concepts, identifying key methodological considerations and establishing adaptations and processes to be undertaken during adaptation of complex population health interventions in new contexts., Ethics and Dissemination: Ethics approval for the Delphi exercise has been obtained from the University of Glasgow and and the RAND institutional research board. Dissemination of the results of this study will be through peer-reviewed publications, workshops at national and international conferences, and a summary of the guidance developed for key organisations and stakeholders., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY. Published by BMJ.)

Objectives: Key stakeholders can have differing views about which information is essential to inform placement decisions for all patients. This study examined consensus across stakeholder groups on the most important individual needs and treatment outcomes for informing decisions specifically about the level of care for an adolescent in substance use treatment., Methods: We conducted an online modified-Delphi process with treatment providers, policymakers, researchers, and parents of adolescents who have received substance use treatment. Participants rated 48 individual needs from the Global Appraisal of Individual Needs-Initial that were mapped onto the 6 dimensions of the American Society of Addiction Medicine Criteria. In addition, participants rated 10 treatment outcomes from the Substance Abuse and Mental Health Services Administration's National Outcome Measures. We assessed consensus within stakeholder groups using the RAND/UCLA Appropriateness Method. We considered the items reaching consensus with the highest ratings across stakeholder groups as the most important individual needs and treatment outcomes., Results: We recruited 194 participants (81 providers, 54 policymakers, 32 researchers, 27 parents). Participants identified suicidality and severity of substance use disorder symptoms as the most important individual needs, and reduction in substance use as the most important treatment outcome., Conclusions: Standardized procedures for matching adolescents to levels of care for substance use treatment should at a minimum be based on assessments of suicidality and severity of substance use disorder symptoms, and consider reduction in substance use as a primary treatment outcome. These findings can progress the development of "level-of-care" decision rules specifically for adolescents.

Background: Maternal weight gain during pregnancy is one of the few potentially modifiable risk factors for many adverse maternal and child health outcomes. Defining the optimal pregnancy weight gain range is difficult because, while lower weight gain may prevent some outcomes, such as maternal and child obesity, it may increase the risk of others such as fetal growth restriction and infant death. These health outcomes vary in their seriousness to mothers and their health care providers, and these differences in seriousness should be taken into account when determining optimal weight gain ranges. However, the relative seriousness that women and their care providers place on different health outcomes is unknown., Objective: We will determine the seriousness of 11 maternal and child health outcomes that have been consistently associated with pregnancy weight gain. We will achieve this by engaging patients and maternal and child health professionals using an online modified Delphi panel process., Methods: We aim to recruit a racially/ethnically and geographically diverse group of 90 US maternal and child health professionals and 90 women who are pregnant or less than 2 years postpartum. We will conduct 3 concurrent panels using the ExpertLens system, a previously evaluated online modified Delphi system that combines 2 rounds of rating with 1 round of feedback and moderated online discussion. In Round 1, panelists are asked to rate the seriousness of each health outcome on a scale of 0-100 and to provide a rationale for their scores. In Round 2, panelists will review their responses relative to those of other panelists. They will discuss their seriousness ratings anonymously using a moderated online discussion board. In Round 3, participants will revise their Round 1 responses based on group feedback and discussion. Each round will be open for 1-2 weeks., Results: The study protocol was reviewed by our ethics boards and did not require approval as human research. A pilot study of 6 professionals and 7 patients was completed in December 2019., Conclusions: Our numeric estimates of the seriousness of maternal and child health outcomes will enable future studies to determine pregnancy weight gain ranges that balance the risks of low and high weight gain for mothers and children., International Registered Report Identifier (irrid): DERR1-10.2196/16478., (©Lisa M Bodnar, Dmitry Khodyakov, Katherine P Himes, Jessica G Burke, Sara Parisi, Jennifer A Hutcheon. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 02.06.2020.)

The development and use of large and dynamic health data repositories designed to support research pose challenges to traditional informed consent models. We used semi-structured interviewing (n=44) to elicit diverse research stakeholders' views of a model of consent appropriate to participation in initiatives that entail collection, long-term storage, and undetermined future research use of multiple types of health data. We demonstrate that, when considering health data repositories, research stakeholders replace a concept of consent as informed with one in which consent is engaged. In engaged consent, a participant's ongoing relationship with a repository serves as a substitute or adjunct to information exchange at enrollment. We detail research stakeholders' views of the risks of engaged consent and suggest questions for further study about engagement and consent procedures in initiatives that aim to store data for future unspecified research purposes.

Patients and caregivers are increasingly recognized as key stakeholders in developing clinical practice guidelines (CPGs). Online engagement approaches offer the promise of a rigorous, scalable, and convenient engagement method. This paper illustrates how an online modified-Delphi approach could be used to engage patients, caregivers, and other stakeholder in CPG development. It provides practical guidance for conducting online modified-Delphi panels that covers (1) joint development of the engagement approach with relevant stakeholders, (2) adaptation of methods used by experts in guideline development, (3) pilot testing, (4) participant recruitment, (5) determining panel size and composition, (6) building participant capacity, (7) facilitation of two-way interaction, (8) continuous engagement and retention of participants, (9) rigorous data analysis, (10) evaluation of engagement activities, and (11) result dissemination. The paper is based on a recently completed study about engaging individuals with Duchenne muscular dystrophy (DMD) and their caregivers in determining the patient-centeredness of DMD care guidelines.

Purpose. To determine the patient-centeredness of endocrine and bone health Duchenne muscular dystrophy (DMD) care considerations using the RAND/PPMD Patient-Centeredness Method (RPM), which is a novel, online, modified-Delphi approach to engaging patients and caregivers in clinical guideline development. Methods. We solicited input on the patient-centeredness of care considerations from 28 individuals with DMD and 94 caregivers, randomly assigned to 1 of 2 mixed panels. During a 3-round online modified-Delphi process, participants rated the importance and acceptability of 19 DMD care considerations (round 1), reviewed and discussed the initial results (round 2), and revised their original ratings (round 3). Patient-centeredness was operationalized as importance and acceptability of recommendations. We considered a care consideration to be patient-centered if both panels deemed it important and acceptable. Results. Ninety-five panelists (78%) participated in this study. Of these, 88 (93%) participated in round 1, 74 (78%) in round 2, and 56 (59%) in round 3. Panelists deemed 12 care considerations to be patient-centered: 3 weight management, 3 bone health, 4 vertical growth, and 2 puberty recommendations. Seven care considerations did not meet patient-centeredness criteria. Common reasons were lack of evidence specific to DMD and concerns about insurance coverage, access to treatment, and patient safety. Conclusions. Using the RPM, Duchenne families considered most care considerations to be patient-centered. Besides being clinically appropriate, these considerations are likely to be consistent with the preferences, needs, and values of Duchenne families. While all relevant care considerations should be discussed during patient-provider encounters, those that did not meet patient-centeredness criteria in particular should be carefully considered as part of joint decision making between Duchenne families and their providers. Study Registration: HSRProj 20163126.

Background: There is a growing interest in creating large-scale repositories that store genetic, behavioral, and environmental data for future, unspecified uses. The All of Us Research Program is one example of such a repository. Its participants will get access to their personal data and the results of the studies that used them. However, little is known about what researchers should return to participants and how they should do it in a way that is valuable and meaningful to participants. Methods: To better understand the concept of "return of value" and the practice of returning valuable study information, we conducted semi-structured telephone interviews with 44 stakeholders with diverse perspectives on this topic. All interviews have been transcribed and coded thematically to identify the most salient themes, to explore differences between returning different types of study results, and to describe differences and similarities in perspectives of different stakeholder groups. Results: We found that one size does not fit all when it comes to returning value to participants: the decisions about return of results are affected by participant preferences, researchers' concerns about feasibility, the types of data collected, their level of granularity, and available options for supporting result interpretation. Conclusions: Our findings suggest that the key to operationalizing return of value and to identifying ways to return valuable information to study participants may be to find a point of equilibrium between criteria that may affect usefulness and feasibility. The point of equilibrium may vary by study, by participants' backgrounds and preferences, by their health literacy and access to regular healthcare, and by the resources available to professionals controlling the data. Future studies should explore the factors that determine the point of equilibrium between feasibility and usefulness.

Background: Clinical care guidelines are typically developed by clinicians and researchers. Including patient and caregiver voices in guideline development may help create guidelines that are more useful for patients and consequently improve their guideline adherence. Although there is substantial research on the factors the affect providers' adherence to guidelines, there is less research on the factors that affect patients' compliance with guideline recommendations, especially among those with rare disorders. The purpose of this study is to explore factors that are likely to affect patient/caregiver adherence to endocrine and bone health recommendations for Duchenne Muscular Dystrophy (DMD). To do so, we used qualitative data collected as part of the study designed to develop, implement, and evaluate a new online, modified-Delphi approach to engaging patients with rare diseases and their caregivers in guideline development, using care guidelines for DMD as a case study., Methods: We thematically analyzed qualitative data collected from 95 adults with DMD and their caregivers who participated in at least one round of our online Modified-Delphi panel process. Participants rated and commented on the patient-centeredness of 19 recommendations about vertical growth, weight management, bone health, and delayed puberty included in the 2018 DMD care considerations. Patient-centeredness was operationalized as the importance and acceptability of care recommendations., Results: Thematic analyses revealed six factors that affect guideline adherence from the patient/caregiver perspective: content and format of recommendations, patient and provider characteristics, and social and financial factors., Conclusions: This study used a novel approach to exploring patient and caregiver perspectives on factors that may affect guideline adherence. The six factors identified by DMD patients and caregivers are similar to the factors affecting provider adherence and are not limited to DMD. Understanding consistency between provider- and patient/caregiver-identified barriers to following guideline recommendations can lead to developing more successful interventions for increasing guideline adherence.

Objectives: Value-based insurance design (VBID) lowers cost sharing for high-value healthcare services that are clinically beneficial to patients with certain conditions. In 2017, the Center for Medicare and Medicaid Innovation began a voluntary VBID model test in Medicare Advantage (MA). This article describes insurers' perspectives on the MA VBID model, explores perceived barriers to joining this model, and describes ways to address participation barriers., Study Design: A descriptive, qualitative study., Methods: In spring/summer 2017, we conducted semistructured interviews with 24 representatives of 10 nonparticipating MA insurers to learn why they did not join the model test. We interviewed 73 representatives of 8 VBID-participating insurers about their participation decisions and implementation experiences. All interview data were analyzed thematically., Results: Fewer than 30% of eligible insurers participated in the first 2 years of the model test. The main barriers to entry were a perceived lack of information on VBID in MA, an expectation of low return on investment, concerns over administrative and information technology (IT) hurdles, and model design parameters. Most VBID participants encountered administrative and IT hurdles but overcame them. CMS made changes to the model parameters to increase the uptake., Conclusions: The model uptake was low, and implementation challenges and concerns over VBID effectiveness in the Medicare population were important factors in participation decisions. To increase uptake, CMS could consider providing in-kind implementation assistance to model participants. Nonparticipants may want to incorporate lessons learned from current participants, and insurers should engage their IT departments/vendors early on.

Although clinical practice guidelines (CPGs) provide recommendations for how best to treat a typical patient with a given condition, patients and their representatives are not always engaged in CPG development. Despite the agreement that patient participation may improve the quality and utility of CPGs, there is no systematic, scalable method for engaging patients and their representatives, as well as no consensus on what exactly patients and their representatives should be asked to do during CPG development. To address these gaps, an interdisciplinary team of researchers, patient representatives, and clinicians developed the RAND/PPMD Patient-Centeredness Method (RPM) - a novel online approach to engaging patients and their representatives in CPG development. The RPM is an iterative approach that allows patients and their representatives to provide input by (1) generating ideas; (2) rating draft recommendations on two criteria (importance and acceptability); (3) explaining and discussing their ratings with other participants using online, asynchronous, anonymous, moderated discussion boards, and (4) revising their responses if needed. The RPM was designed to be consistent with the RAND/UCLA Appropriateness Method used by clinicians and researchers to develop CPG, while helping patients and their representative rate outcome importance and recommendation acceptability - two key components of the GRADE Evidence to Decision (EtD) framework. With slight modifications, the RPM has the potential to explore consensus among key stakeholders on other dimensions of the EtD, including feasibility, equity, and resource use.

Background: Patient engagement in clinical practice guideline (CPG) development has increased significantly in recent years. However, only a few patients and caregivers join CPG development groups., Objective: To describe participant experiences with a novel online, scalable approach for patient and caregiver engagement in CPG development., Design: We developed and tested the RAND/PPMD Patient-Centeredness Method (RPM), a novel online modified-Delphi approach to patient engagement in CPG development that consists of an optional idea generation round and two rating rounds interspersed with an online discussion round., Setting and Participants: Using the online ExpertLens™ system, we ran 2 concurrent panels of patients and caregivers of individuals with Duchenne Muscular Dystrophy (DMD)., Main Outcome Measures: We surveyed all 95 panel participants about their participation and satisfaction with the process. We also conducted telephone interviews with 25 participants., Results: Participants expressed satisfaction with various ExpertLens™ features, noting that the system fostered lively interaction among them. Panelists also appreciated participating in an educational, interactive and convenient discussion forum that allowed them to share their opinions on the importance and acceptability of different recommendations. The RPM was viewed as empowering by patients and their caregivers who felt it would be useful for CPG developers., Discussion and Conclusion: The results of our study show the overall participant satisfaction with a novel, scalable, online approach to engaging patients and caregivers in CPG development, which allows them to share their perspectives and lived experiences using a rigorous, systematic and iterative way that is similar to how clinicians provide their input., Competing Interests: Conflicts of Interest All other co-authors report no conflicts of interest.

Introduction/objectives: The engagement of patients and other stakeholders is a critical element in the design of patient-centered outcomes research studies. However, methodology for scalable engagement in research management particularly activities such as operationalization of principles and setting of priorities is not well-developed. The objective of this study is to describe a novel approach for scalable stakeholder engagement in research aligned with the Patient-Centered Outcomes Research Institute (PCORI) engagement principles, which was evaluated in a national clinical data research network., Materials and Methods: Patient, patient advocate, clinician, and researcher stakeholders were recruited from clinical sites, as well as social media sites related to the 3 conditions of focus, heart failure, obesity, and Kawasaki disease. The engagement strategy was designed, implemented, and mapped to the PCORI engagement principles. Evaluation included internal assessment and quantitative measures of online engagement., Results: We operationalized the PCORI principles with 12 stakeholder engagement strategies and convened stakeholder advisory boards and online research prioritization panels to determine research priorities in a rigorous, deliberative process. A total of 46 advisors (20 patients) and 339 panelists (159 patients) actively participated. There were not significant differences between patients and clinicians in level of online engagement. Nonetheless, while patients reported a slightly greater challenge with following online discussion, they overall had a more favorable opinion about use of the online format., Discussion/conclusion: An efficient way to engage large numbers of representative stakeholders in research is a necessary first step to assure the public of trustworthy use of data networks for health research. This paper describes a comprehensive approach to engagement in patient-centered outcomes research management that informs ongoing development of rigorous methodologies in this area.

Objective: Depressed individuals may require help from different agencies to address health and social needs, but how such coordination occurs in under-resourced communities is poorly understood. This study sought to identify priorities of Latino and African American depressed clients, explore whether service providers understand client priorities, and describe how providers address them., Methods: Between October 2014 and February 2015, we interviewed 104 clients stratified by depression history and 50 representatives of different programs in health and social community agencies who participated in Community Partners in Care, a cluster-randomized trial of coalition-building approaches to delivering depression quality improvement programs. Clients were queried about their most pressing needs; program representatives identified their clients' needs and explained how they addressed them., Results: Physical and mental health were clients' top priorities, followed by housing, caring for and building relationships with others, and employment. While persistently depressed clients prioritized mental health, those with improved depression prioritized relationships with others. Program representatives identified housing, employment, mental health, and improving relationships with others as clients' top priorities. Needs assessment, client-centered services, and linkages to other agencies were main strategies used to address client needs., Conclusion: Depressed clients have multiple health and social needs, and program representatives in under-resourced communities understand the complexity of clients' needs. Agencies rely on needs assessment and referrals to meet their clients' needs, which enhances the importance of agency partnership in "whole person" initiatives. Our results illustrate agency capacity to adopt integrated care models that will address clients' multiple needs through multi-sector collaboration and describe potential strategies to help reach the goal of whole person care., Competing Interests: Competing Interests: None declared.

Objectives: To empirically demonstrate how undisclosed analytic flexibility provides substantial latitude for data mining and selective reporting of consensus in Delphi processes., Study Design and Setting: Pooling data across eight online modified-Delphi panels, we first calculated the percentage of items reaching consensus according to descriptive analysis procedures commonly used in health research but selected post hoc in this article. We then examined the variability of items reaching consensus across panels., Results: Pooling all panel data, the percentage of items reaching consensus ranged from 0% to 84%, depending on the analysis procedure. Comparing data across panels, variability in the percentage of items reaching consensus for each analysis procedure ranged from 0 (i.e., all panels had the same percentage of items reaching consensus for a given analysis procedure) to 83 (i.e., panels had a range of 11% to 94% of items reaching consensus for a given analysis procedure). Of 200 total panel-by-analysis-procedure configurations, four configurations (2%) had all items and 64 (32%) had no items reaching consensus., Conclusion: Undisclosed analytic flexibility makes it unacceptably easy to data mine for and selectively report consensus in Delphi processes. As a solution, we recommend prospective, complete registration of preanalysis plans for consensus-oriented Delphi processes in health research., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Clinical practice guidelines (CPGs) have been widely used in healthcare policy, practice, and for suggesting future research. As patients increasingly become involved in CPG development to produce patient-centered recommendations, more research is needed on methods to engage patients, particularly methods allowing for scalable engagement of large, diverse, and geographically distributed groups of patients. In this article, we discuss practical considerations for using online methods to engage patients in CPG development. To inform this discussion, we conducted a rapid, systematic review of literature on patient involvement in CPG development and used qualitative evidence synthesis techniques to make inferences about potential advantages and challenges of using online methods to engage patients in this context. We identified 79 articles containing information about involving patients in CPG development. Potential advantages include the ability of online methods to facilitate greater openness and honesty by patients, as well as to reflect the diversity of patient views, which in turn further improve the utility of CPGs. Potential challenges of using online methods may include the extra skill, time, and certain types of resources that may be needed for patient engagement, as well as the difficulty engaging specific patient populations. However, these challenges are mitigated by growing calls for patient engagement as normative for CPG development in addition to patients' increasing familiarity with online technologies. These practical considerations should be examined empirically as guideline development groups further explore the appropriateness of using online methods to engage patients across different stages of CPG development.

Objective: The widely used Official Disability Guidelines (ODG), a utilization review guideline for occupational conditions, has not been independently evaluated recently., Methods: We applied the appraisal of guidelines for research and evaluation (AGREE II) and modified a measurement tool to assess systematic reviews (AMSTAR) instruments to assess guideline development methods and the quality of supporting systematic reviews. Multidisciplinary experts rated the validity of clinical content for 47 topics., Results: The overall AGREE II score was 58% due to a combination of favorable attributes (breadth, clear recommendations, frequent updating, and application tools) and unfavorable attributes (scant input from workers and uncertainty about editorial independence). The modified AMSTAR rating was fair/good due to limited information on methods. Panelists rated clinical content as valid for 41 topics., Conclusions: ODG appears to be acceptable to clinicians, but ODG requires greater rigor to keep pace with methodological advances in the field of guideline development.

Background: Children with medical complexity (CMC) account for disproportionate hospital utilization and adverse outcomes after discharge, and several gaps exist regarding the quality of hospital to home transitional care for this population. We conducted an expert elicitation process to identify important and feasible hospital to home transitional care interventions for CMC from the perspectives of parents and health care professionals., Methods: We conducted a 2-round electronic Delphi process to identify important and feasible transitional care interventions. Panelists included parents of CMC and multidisciplinary health care professionals. In the first round, panelists rated the importance and feasibility of 39 transitional care interventions on a 9-point Likert scale; agreement between panelists was defined according to RAND/UCLA Appropriateness Methods. The second round of data collection evaluated 16 interventions that panelists did not agree on in the first round and 8 new or revised interventions, accompanied by quantitative and qualitative data summaries., Results: A total of 29 parents of CMC and 37 health care professionals participated in the Delphi process (response rate 75%). Both stakeholder panels endorsed most interventions as important; health care professionals were less likely to rate several interventions as feasible compared with the parent panel. Over 2 rounds of data collection, the 2 stakeholder panels endorsed 25 interventions as important as well as feasible. These interventions related to family engagement during the hospitalization, care coordination and social support assessment, predischarge education, and written materials., Conclusions: Parents and health care professionals considered several transitional care interventions important as well as feasible. This research might inform hospitals' transitional care programs and policies., (Copyright © 2017 Academic Pediatric Association. Published by Elsevier Inc. All rights reserved.)

Participatory researchers rely on relationship-based ethical considerations to address ethical challenges, but little is known about how these considerations shape day-to-day decision-making. We presented vignettes describing ethical dilemmas to health researchers with varying degrees of experience in participatory research in an online modified-Delphi panel. Panelists rated potential solutions to dilemmas and provided rationales for choosing solutions in text boxes and discussion boards. Panelists agreed on a preferred solution for three of five vignettes. Panelists indicated that ethical research decision-making should be inclusive of diverse perspectives. Panelists prioritized scientific goals above participatory ones and viewed tenets of scientific validity as value-free. Panelists indicated that participatory research ethics would be difficult to codify, but that vignette-based learning could support didactic objectives.