Search

Background and Objective: Although comprehensive and widespread guidelines on how to conduct systematic reviews of outcome measurement instruments (OMIs) exist, for example from the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) initiative, key information is often missing in published reports. This article describes the development of an extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guideline: PRISMA-COSMIN for OMIs 2024., Methods: The development process followed the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) guidelines and included a literature search, expert consultations, a Delphi study, a hybrid workgroup meeting, pilot testing, and an end-of-project meeting, with integrated patient/public involvement., Results: From the literature and expert consultation, 49 potentially relevant reporting items were identified. Round 1 of the Delphi study was completed by 103 panelists, whereas round 2 and 3 were completed by 78 panelists. After 3 rounds, agreement (≥67%) on inclusion and wording was reached for 44 items. Eleven items without consensus for inclusion and/or wording were discussed at a workgroup meeting attended by 24 participants. Agreement was reached for the inclusion and wording of 10 items, and the deletion of 1 item. Pilot testing with 65 authors of OMI systematic reviews further improved the guideline through minor changes in wording and structure, finalized during the end-of-project meeting. The final checklist to facilitate the reporting of full systematic review reports contains 54 (sub)items addressing the review's title, abstract, plain language summary, open science, introduction, methods, results, and discussion. Thirteen items pertaining to the title and abstract are also included in a separate abstract checklist, guiding authors in reporting for example conference abstracts., Conclusion: PRISMA-COSMIN for OMIs 2024 consists of two checklists (full reports; abstracts), their corresponding explanation and elaboration documents detailing the rationale and examples for each item, and a data flow diagram. PRISMA-COSMIN for OMIs 2024 can improve the reporting of systematic reviews of OMIs, fostering their reproducibility and allowing end-users to appraise the quality of OMIs and select the most appropriate OMI for a specific application. NOTE: This paper was jointly developed by Journal of Clinical Epidemiology, Quality of Life Research, Journal of Patient Reported Outcomes, Health and Quality of Life Outcomes and jointly published by Elsevier Inc, Springer Nature Switzerland AG, and BioMed Central Ltd., part of Springer Nature. The articles are identical except for minor stylistic and spelling differences in keeping with each journal's style. Either citation can be used when citing this article., Competing Interests: Declaration of competing interest Drs Terwee and Mokkink are the founders of COSMIN; Dr Moher is a member of the PRISMA executive; Dr Tricco is the co-editor in chief of the Journal of Clinical Epidemiology. The other authors have no conflicts of interest., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objectives: We sought to assess and report harms that were observed but not disclosed previously in clinical trials of gabapentin., Study Design and Setting: We reanalyzed individual participant data from six randomized parallel trials of gabapentin for neuropathic pain, and we conducted meta-analyses. Between 1996 and 2003, adult participants were assigned to gabapentin (600 mg-3,600 mg per day) or placebo for 7-14 weeks. We calculated the proportion of participants with: one or more adverse events (AEs), one or more serious AEs, discontinued, discontinued because AEs. We also estimated effects on AEs at three levels of aggregation using COSTART, a hierarchical system for classifying AEs: body system, midlevel system, preferred term., Results: We found evidence of important harms that were neither included in published trial reports nor included in systematic reviews. Aggregating related harms led to greater confidence that gabapentin might harm the nervous system and possibly the digestive, metabolic and nutritional, respiratory, sensory, and urogenital body systems. Nervous system harms were more common than previous reports suggest., Conclusion: Clinical trials identified harms that were not reported publicly, and journal articles overstated uncertainty about harms. Relying on journal articles to evaluate gabapentin's harms could contribute to incomplete and misleading conclusions in systematic reviews and guidelines. To prevent bias arising from the selective nonreporting of results, journal articles should describe how to access data for all harms observed in clinical trials (e.g., by sharing individual participant data)., Competing Interests: Declaration of competing interest Evan Mayo-Wilson and Tianjing Li previously received support for research about gabapentin from the Patient Centered Outcomes Research Institute (PCORI). All other authors declare they have no competing interests., (Copyright © 2023 Elsevier Inc. All rights reserved.)

Objectives: To describe randomized controlled trial (RCT) methodology reporting over time., Study Design and Setting: We used a deep learning-based sentence classification model based on the Consolidated Standards of Reporting Trials (CONSORT) statement, considered minimum requirements for reporting RCTs. We included 176,469 RCT reports published between 1966 and 2018. We analyzed the reporting trends over 5-year time periods, grouping trials from 1966 to 1990 in a single stratum. We also explored the effect of journal impact factor (JIF) and medical discipline., Results: Population, Intervention, Comparator, Outcome (PICO) items were commonly reported during each period, and reporting increased over time (e.g., interventions: 79.1% during 1966-1990 to 87.5% during 2010-2018). Reporting of some methods information has increased, although there is room for improvement (e.g., sequence generation: 10.8-41.8%). Some items are reported infrequently (e.g., allocation concealment: 5.1-19.3%). The number of items reported and JIF are weakly correlated (Pearson's r (162,702) = 0.16, P < 0.001). The differences in the proportion of items reported between disciplines are small (<10%)., Conclusion: Our analysis provides large-scale quantitative support for the hypothesis that RCT methodology reporting has improved over time. Extending these models to all CONSORT items could facilitate compliance checking during manuscript authoring and peer review, and support metaresearch., Competing Interests: Declaration of competing interest H.K., L.J., L.H., and E.M.-W. received funding from the National Library Of Medicine of the National Institutes of Health under the award number R01LM014079. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funder had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication., (Copyright © 2023 Elsevier Inc. All rights reserved.)

Randomized controlled trials remain the reference standard for healthcare research on effects of interventions, and the need to report both benefits and harms is essential. The Consolidated Standards of Reporting Trials (the main CONSORT) statement includes one item on reporting harms (i.e., all important harms or unintended effects in each group). In 2004, the CONSORT group developed the CONSORT Harms extension; however, it has not been consistently applied and needs to be updated. Here, we describe CONSORT Harms 2022, which replaces the CONSORT Harms 2004 checklist, and shows how CONSORT Harms 2022 items could be incorporated into the main CONSORT checklist. Thirteen items from the main CONSORT were modified to improve harms reporting. Three new items were added. In this article, we describe CONSORT Harms 2022 and how it was integrated into the main CONSORT checklist and elaborate on each item relevant to complete reporting of harms in randomized controlled trials. Until future work from the CONSORT group produces an updated checklist, authors, journal reviewers, and editors of randomized controlled trials should use the integrated checklist presented in this paper., (Copyright © 2023. Published by Elsevier Inc.)

The clinical guidelines that underpin the use of drugs for mental disorders are informed by evidence from randomised controlled trials (RCTs). RCTs are performed to obtain marketing authorisation from regulators. The methods used in these RCTs could be appropriate for early phases of drug development because they identify drugs with important harms and drugs that are efficacious for specific health problems and populations. RCTs done before marketing authorisation do not tend to address clinical questions that concern the effectiveness of a drug in heterogeneous and comorbid populations, the optimisation of drug sequencing and discontinuation, or the comparative benefits and harms of different drugs that could be used for the same health problem. This Review proposes an overview of some shortcomings of RCTs, at an individual level and at the whole portfolio level, and identifies some methods in planning, conducting, and carrying out analyses in RCTs that could enhance their ability to support therapeutic decisions. These suggestions include: identifying patient-important questions to be investigated by psychopharmacological RCTs; embedding pragmatic RCTs within clinical practice to improve generalisability to target populations; collecting evidence about drugs in overlooked populations; developing methods to facilitate the recruitment of patients with mental disorders and to reduce the number of patients who drop out, using specific methods; using core outcome sets to standardise the assessment of benefits and harms; and recording systematically serious objective outcomes, such as suicide or hospitalisation, to be evaluated in meta-analyses. This work is a call to address questions relevant to patients using diverse design of RCTs, thus contributing to the development of a patient-centred, evidence-based psychiatry., Competing Interests: Declaration of interests EMW declared grants from the Patient Centered Outcomes Research Institute, the US Food and Drug Administration, Arnold Ventures, the Sloan Foundation, The US National Institutes of Health (NIH), the Robert Wood Johnson Foundation, and the American Public Health Association. EM-W received payment for teaching at a summer institute in the NIH. TAF declared a grant, licenses, and consulting fees from Mitsubishi-Tanabe and Shinogi. TAF received payment for lectures from Merck. TAF also has issued or has pending patents from Tanabe-Mitsubishi. AC, PR, and VC declare no competing interests., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

Objective: We compared methods used with current recommendations for synthesizing harms in systematic reviews and meta-analyses (SRMAs) of gabapentin., Study Design & Setting: We followed recommended systematic review practices. We selected reliable SRMAs of gabapentin (i.e., met a pre-defined list of methodological criteria) that assessed at least one harm. We extracted and compared methods in four areas: pre-specification, searching, analysis, and reporting. Whereas our focus in this paper is on the methods used, Part 2 examines the results for harms across reviews., Results: We screened 4320 records and identified 157 SRMAs of gabapentin, 70 of which were reliable. Most reliable reviews (51/70; 73%) reported following a general guideline for SRMA conduct or reporting, but none reported following recommendations specifically for synthesizing harms. Across all domains assessed, review methods were designed to address questions of benefit and rarely included the additional methods that are recommended for evaluating harms., Conclusion: Approaches to assessing harms in SRMAs we examined are tokenistic and unlikely to produce valid summaries of harms to guide decisions. A paradigm shift is needed. At a minimal, reviewers should describe any limitations to their assessment of harms and provide clearer descriptions of methods for synthesizing harms., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)

Objective: Most systematic reviews of interventions focus on potential benefits. Common methods and assumptions that are appropriate for assessing benefits can be inappropriate for harms. This paper provides a primer on researching harms, particularly in systematic reviews., Study Design and Setting: Commentary describing challenges with assessing harm., Results: Investigators should be familiar with various terminologies used to describe, classify, and group harms. Published reports of clinical trials include limited information about harms, so systematic reviewers should not depend on these studies and journal articles to reach conclusions about harms. Visualizations might improve communication of multiple dimensions of harms such as severity, relatedness, and timing., Conclusion: The terminology, classification, detection, collection, and reporting of harms create unique challenges that take time, expertise, and resources to navigate in both primary studies and evidence syntheses. Systematic reviewers might reach incorrect conclusions if they focus on evidence about harms found in published reports of randomized trials of a particular health problem. Systematic reviews could be improved through better identification and reporting of harms in primary studies and through better training and uptake of appropriate methods for synthesizing evidence about harms., (Copyright © 2021. Published by Elsevier Inc.)

Objective: In this methodologic study (Part 2 of 2), we examined the overlap in sources of evidence and the corresponding results for harms in systematic reviews for gabapentin., Study Design & Setting: We extracted all citations referenced as sources of evidence for harms of gabapentin from 70 systematic reviews, as well as the harms assessed and numerical results. We assessed consistency of harms between pairs of reviews with a high degree of overlap in sources of evidence (>50%) as determined by corrected covered area (CCA)., Results: We found 514 reports cited across 70 included reviews. Most reports (244/514, 48%) were not cited in more than one review. Among 18 pairs of reviews, we found reviews had differences in which harms were assessed and their choice to meta-analyze estimates or present descriptive summaries. When a specific harm was meta-analyzed in a pair of reviews, we found similar effect estimates., Conclusion: Differences in harms results across reviews can occur because the choice of harms is driven by reviewer preferences, rather than standardized approaches to selecting harms for assessment. A paradigm shift is needed in the current approach to synthesizing harms., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews., (Copyright © 2021. Published by Elsevier Inc.)

Objective: We examined nonsystematic adverse events (AEs) in Part 2 (of 2) of a study describing the assessment and reporting AEs in clinical trials., Study Design and Setting: We examined 21 trials of gabapentin for neuropathic pain (52 sources) and seven trials of quetiapine for bipolar depression (80 sources) using data from the Multiple Data Sources study. We extracted and compared information about nonsystematic AEs (i.e., AEs that were not assessed for every participant), including AEs categorized as "serious." We recorded whether AEs were grouped by anatomic or physiological system., Results: Trials of the same drug reported information about different AEs. Information in public sources was inadequate for decision-making. No public source reported all AEs, or all serious AEs, identified in nonpublic sources about the same trial. Of trials with only public sources, 2/15 (13%) gabapentin and 0/3 (0%) quetiapine trials grouped AEs by anatomic or physiological system., Conclusion: Public sources contained little information about nonsystematic AEs, including serious AEs. Grouping might make nonsystematic AEs easier to detect; however, most public sources did not report grouped AEs. Standards are needed to improve the collection and reporting of nonsystematic AEs so that stakeholders can use trials to assess the balance of potential benefits and harms., (Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: We examined systematic adverse events (AEs) in Part 1 (of 2) of a study describing the assessment and reporting of AEs in clinical trials., Study Design and Setting: We examined 52 public and nonpublic data sources about trials of quetiapine for bipolar depression using data from the Multiple Data Sources study. We extracted and compared information about systematic AEs (i.e., AEs assessed for all participants) in six prespecified domains: cardiovascular, cholesterol, endocrine, extrapyramidal symptoms, mania, and weight., Results: Eligible trials did not assess and report the same systematic AEs, and most results were not available in public sources. Overall, public sources reported 159 results, of which 92 of 159 (58%) included sufficient statistical information to calculate the treatment effect and its precision. Nonpublic sources reported 636 results; 630 of 636 (99%) reported sufficient statistical information., Conclusion: Systematic AEs were defined and analyzed in many ways, which led to many numerical results. Most systematic AEs were not mentioned in public sources. To minimize bias, methods for defining and analyzing potential AEs should be prespecified in trial registers and protocols. All trial results should be publicly available so that stakeholders can compare benefits and AEs. Trials could report core sets of AEs to facilitate decision-making., (Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: To provide recommendations for the selection of comparators for randomized controlled trials of health-related behavioral interventions., Study Design and Setting: The National Institutes of Health Office of Behavioral and Social Science Research convened an expert panel to critically review the literature on control or comparison groups for behavioral trials and to develop strategies for improving comparator choices and for resolving controversies and disagreements about comparators., Results: The panel developed a Pragmatic Model for Comparator Selection in Health-Related Behavioral Trials. The model indicates that the optimal comparator is the one that best serves the primary purpose of the trial but that the optimal comparator's limitations and barriers to its use must also be taken into account., Conclusion: We developed best practice recommendations for the selection of comparators for health-related behavioral trials. Use of the Pragmatic Model for Comparator Selection in Health-Related Behavioral Trials can improve the comparator selection process and help resolve disagreements about comparator choices., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Objectives: The objective of this study was to determine whether disagreements among multiple data sources affect systematic reviews of randomized clinical trials (RCTs)., Study Design and Setting: Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic sources (clinical study reports [CSRs] and individual participant data [IPD])., Results: We found 21 gabapentin RCTs (74 reports, 6 IPD) and 7 quetiapine RCTs (50 reports, 1 IPD); most were reported in journal articles (18/21 [86%] and 6/7 [86%], respectively). When available, CSRs contained the most trial design and risk of bias information. CSRs and IPD contained the most results. For the outcome domains "pain intensity" (gabapentin) and "depression" (quetiapine), we found single trials with 68 and 98 different meta-analyzable results, respectively; by purposefully selecting one meta-analyzable result for each RCT, we could change the overall result for pain intensity from effective (standardized mean difference [SMD] = -0.45; 95% confidence interval [CI]: -0.63 to -0.27) to ineffective (SMD = -0.06; 95% CI: -0.24 to 0.12). We could change the effect for depression from a medium effect (SMD = -0.55; 95% CI: -0.85 to -0.25) to a small effect (SMD = -0.26; 95% CI: -0.41 to -0.1)., Conclusions: Disagreements across data sources affect the effect size, statistical significance, and interpretation of trials and meta-analyses., (Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.)

Objective: To identify variations in outcomes and results across reports of randomized clinical trials (RCTs)., Study Design and Setting: Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic (e.g., clinical study reports) sources by 2015. We prespecified outcome domains. From each source, we collected "outcomes" (i.e., domain, measure, metric, method of aggregation, and time point); "treatment effect" (i.e., outcome plus the methods of analysis [e.g., how missing data were handled]); and results (i.e., numerical contrasts of treatment and comparison groups). We assessed whether results included sufficient information for meta-analysis., Results: We found 21 gabapentin (68 public, 6 nonpublic reports) and seven quetiapine RCTs (46 public, 4 nonpublic reports). For four (gabapentin) and seven (quetiapine) prespecified outcome domains, RCTs reported 214 and 81 outcomes by varying four elements. RCTs assessed 605 and 188 treatment effects by varying the analysis of those outcomes. RCTs reported 1,230 and 661 meta-analyzable results, 305 (25%) and 109 (16%) in public reports., Conclusion: RCTs included hundreds of outcomes and results; a small proportion were in public reports. Trialists and meta-analysts may cherry-pick what they report from multiple sources of RCT information., (Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.)

Background: Social anxiety disorder-a chronic and naturally unremitting disease that causes substantial impairment-can be treated with pharmacological, psychological, and self-help interventions. We aimed to compare these interventions and to identify which are most effective for the acute treatment of social anxiety disorder in adults., Methods: We did a systematic review and network meta-analysis of interventions for adults with social anxiety disorder, identified from published and unpublished sources between 1988 and Sept 13, 2013. We analysed interventions by class and individually. Outcomes were validated measures of social anxiety, reported as standardised mean differences (SMDs) compared with a waitlist reference. This study is registered with PROSPERO, number CRD42012003146., Findings: We included 101 trials (13 164 participants) of 41 interventions or control conditions (17 classes) in the analyses. Classes of pharmacological interventions that had greater effects on outcomes compared with waitlist were monoamine oxidase inhibitors (SMD -1·01, 95% credible interval [CrI] -1·56 to -0·45), benzodiazepines (-0·96, -1·56 to -0·36), selective serotonin-reuptake inhibitors and serotonin-norepinephrine reuptake inhibitors (SSRIs and SNRIs; -0·91, -1·23 to -0·60), and anticonvulsants (-0·81, -1·36 to -0·28). Compared with waitlist, efficacious classes of psychological interventions were individual cognitive-behavioural therapy (CBT; SMD -1·19, 95% CrI -1·56 to -0·81), group CBT (-0·92, -1·33 to -0·51), exposure and social skills (-0·86, -1·42 to -0·29), self-help with support (-0·86, -1·36 to -0·36), self-help without support (-0·75, -1·25 to -0·26), and psychodynamic psychotherapy (-0·62, -0·93 to -0·31). Individual CBT compared with psychological placebo (SMD -0·56, 95% CrI -1·00 to -0·11), and SSRIs and SNRIs compared with pill placebo (-0·44, -0·67 to -0·22) were the only classes of interventions that had greater effects on outcomes than appropriate placebo. Individual CBT also had a greater effect than psychodynamic psychotherapy (SMD -0·56, 95% CrI -1·03 to -0·11) and interpersonal psychotherapy, mindfulness, and supportive therapy (-0·82, -1·41 to -0·24)., Interpretation: Individual CBT (which other studies have shown to have a lower risk of side-effects than pharmacotherapy) is associated with large effect sizes. Thus, it should be regarded as the best intervention for the initial treatment of social anxiety disorder. For individuals who decline psychological intervention, SSRIs show the most consistent evidence of benefit., Funding: National Institute for Health and Care Excellence., (Copyright © 2014 Mayo-Wilson et al. Open Access article distributed under the terms of CC BY. Published by Elsevier Ltd.. All rights reserved.)

Objectives: This article presents a new tool that helps systematic reviewers to extract and compare implementation data across primary trials. Currently, systematic review guidance does not provide guidelines for the identification and extraction of data related to the implementation of the underlying interventions., Study Design and Setting: A team of systematic reviewers used a multistaged consensus development approach to develop this tool. First, a systematic literature search on the implementation and synthesis of clinical trial evidence was performed. The team then met in a series of subcommittees to develop an initial draft index. Drafts were presented at several research conferences and circulated to methodological experts in various health-related disciplines for feedback. The team systematically recorded, discussed, and incorporated all feedback into further revisions. A penultimate draft was discussed at the 2010 Cochrane-Campbell Collaboration Colloquium to finalize its content., Results: The Oxford Implementation Index provides a checklist of implementation data to extract from primary trials. Checklist items are organized into four domains: intervention design, actual delivery by trial practitioners, uptake of the intervention by participants, and contextual factors. Systematic reviewers piloting the index at the Cochrane-Campbell Colloquium reported that the index was helpful for the identification of implementation data., Conclusion: The Oxford Implementation Index provides a framework to help reviewers assess implementation data across trials. Reviewers can use this tool to identify implementation data, extract relevant information, and compare features of implementation across primary trials in a systematic review. The index is a work-in-progress, and future efforts will focus on refining the index, improving usability, and integrating the index with other guidance on systematic reviewing., (Copyright © 2013 Elsevier Inc. All rights reserved.)