Search

Background: Systematic reviews (SRs) are used to inform clinical practice guidelines and healthcare decision making by synthesising the results of primary studies. Efficiently retrieving as many relevant SRs as possible is challenging with a minimum number of databases, as there is currently no guidance on how to do this optimally. In a previous study, we determined which individual databases contain the most SRs, and which combination of databases retrieved the most SRs. In this study, we aimed to validate those previous results by using a different, larger, and more recent set of SRs., Methods: We obtained a set of 100 Overviews of Reviews that included a total of 2276 SRs. SR inclusion was assessed in MEDLINE, Embase, and Epistemonikos. The mean inclusion rates (% of included SRs) and corresponding 95% confidence intervals were calculated for each database individually, as well as for combinations of MEDLINE with each other database and reference checking. Features of SRs not identified by the best database combination were reviewed qualitatively., Results: Inclusion rates of SRs were similar in all three databases (mean inclusion rates in % with 95% confidence intervals: 94.3 [93.9-94.8] for MEDLINE, 94.4 [94.0-94.9] for Embase, and 94.4 [93.9-94.9] for Epistemonikos). Adding reference checking to MEDLINE increased the inclusion rate to 95.5 [95.1-96.0]. The best combination of two databases plus reference checking consisted of MEDLINE and Epistemonikos (98.1 [97.7-98.5]). Among the 44/2276 SRs not identified by this combination, 34 were published in journals from China, four were other journal publications, three were health agency reports, two were dissertations, and one was a preprint. When discounting the journal publications from China, the SR inclusion rate in the recommended combination (MEDLINE, Epistemonikos and reference checking) was even higher than in the previous study (99.6 vs. 99.2%)., Conclusions: A combination of databases and reference checking was the best approach to searching for biomedical SRs. MEDLINE and Epistemonikos, complemented by checking the references of the included studies, was the most efficient and produced the highest recall. However, our results point to the presence of geographical bias, because some publications in journals from China were not identified., Study Registration: https://doi.org/10.17605/OSF.IO/R5EAS (Open Science Framework)., (© 2024. The Author(s).)

The strength of a systematic review hinges in large part on the quality of evidence it appraises. Observational studies, including cohort studies and case series, are positioned lower on the hierarchy of evidence, with cohort studies typically higher than case series. The often subtle differences between these study designs may lead to misclassification and can impact the strength of recommendations derived from such data. This manuscript offers an approach to delineate the differences between cohort studies and case series and provides clinical examples of these subtleties. By providing a simple approach for distinguishing cohort studies from case series, this manuscript seeks to assist researchers performing systematic reviews., (© 2024. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)

Background: Assessing the methodological quality of case reports and case series is challenging due to human judgment variability and time constraints. We evaluated the agreement in judgments between human reviewers and GPT-4 when applying a standard methodological quality assessment tool designed for case reports and series., Methods: We searched Scopus for systematic reviews published in 2023-2024 that cited the appraisal tool by Murad et al. A GPT-4 based agent was developed to assess the methodological quality using the 8 signaling questions of the tool. Observed agreement and agreement coefficient were estimated comparing published judgments of human reviewers to GPT-4 assessment., Results: We included 797 case reports and series. The observed agreement ranged between 41.91% and 80.93% across the eight questions (agreement coefficient ranged from 25.39 to 79.72%). The lowest agreement was noted in the first signaling question about selection of cases. The agreement was similar in articles published in journals with impact factor < 5 vs. ≥ 5, and when excluding systematic reviews that did not use 3 causality questions. Repeating the analysis using the same prompts demonstrated high agreement between the two GPT-4 attempts except for the first question about selection of cases., Conclusions: The study demonstrates a moderate agreement between GPT-4 and human reviewers in assessing the methodological quality of case series and reports using the Murad tool. The current performance of GPT-4 seems promising but unlikely to be sufficient for the rigor of a systematic review and pairing the model with a human reviewer is required., (© 2024. The Author(s).)

Background: The current practice in guideline development is to use the control group event rate (CR) as a surrogate of baseline risk and to assume portability of the relative treatment effect across populations with low, moderate and high baseline risk. We sought to emulate this practice in a very large sample of meta-analyses., Methods: We retrieved data from all meta-analyses published in the Cochrane Database of Systematic Reviews (2003-2020) that evaluated a binary outcome, reported 2 × 2 data for each individual study and included at least 4 studies. We excluded studies with no events. We conducted meta-analyses with odds ratios and relative risks and performed subgroup analyses based on tertiles of CR. In sensitivity analyses, we evaluated the use of total event rate (TR) instead of CR and using quartiles instead of tertiles., Results: The analysis included 2,531 systematic reviews (27,692 meta-analyses, 226,975 studies, 25,669,783 patients).The percentages of meta-analyses with statistically significant interaction (P < 0.05) based on CR tertile or quartile ranged 12-18% across various sensitivity analyses. This percentage increased as the number of studies or range of CR per meta-analysis increased, reflecting increased power of the subgroup test. The percentages of meta-analyses with statistically significant interaction (P < 0.05) with TR quantiles were lower than those with CR but remained higher than expected by chance., Conclusion: This analysis suggests that when CR or TR are used as surrogates for baseline risk, relative treatment effects may not be portable across populations with varying baseline risks in many meta-analyses. Categroization of the continuous CR variable and not addressing measurement error limit inferences from such analyses and imply that CR is an undesirable source for baseline risk. Guideline developers and decision-makers should be provided with relative and absolute treatment effects that are conditioned on the baseline risk or derived from studies with similar baseline risk to their target populations., (© 2024. The Author(s).)

In response to the evolving needs of knowledge synthesis, this manuscript introduces the concept of narrative reanalysis, a method that refines data from initial reviews, such as systematic and reviews, to focus on specific sub-phenomena. Unlike traditional narrative reviews, which lack the methodological rigor of systematic reviews and are broader in scope, our methodological framework for narrative reanalysis applies a structured, systematic framework to the interpretation of existing data. This approach enables a focused investigation of nuanced topics within a broader dataset, enhancing understanding and generating new insights. We detail a five-stage methodological framework that guides the narrative reanalysis process: (1) retrieval of an initial review, (2) identification and justification of a sub-phenomenon, (3) expanded search, selection, and extraction of data, (4) reanalyzing the sub-phenomenon, and (5) writing the report. The proposed framework aims to standardize narrative reanalysis, advocating for its use in academic and research settings to foster more rigorous and insightful literature reviews. This approach bridges the methodological gap between narrative and systematic reviews, offering a valuable tool for researchers to explore detailed aspects of broader topics without the extensive resources required for systematic reviews., (© 2024 The Author(s). Research Synthesis Methods published by John Wiley & Sons Ltd.)

Background: Initiated in 2021, a Brazilian project aims to establish a national injury registry, compiling comprehensive data on events and individuals across the country, irrespective of injury severity. The registry integrates information from prehospital and hospital care, diverse health systems lacking interoperability, and sectors such as firefighters and the police. Its primary goal is to enhance health surveillance by providing timely, high-quality information, guiding prevention strategies, and informing policy making. The project still aims to reduce long-term morbidity and mortality associated with injuries., Objective: A knowledge gap remains regarding the effects of injury registries in relation to policies and injury outcomes. This protocol outlines a systematic review and meta-analysis to answer "What is the effect of implementation and use of injury registry data on policy making, hospitalization, and mortality?", Methods: The systematic review follows PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, focusing on studies reporting results related to the implementation and use of injury registries, including trauma registries. Outcomes of interest include policy making, hospitalization rates or duration, and mortality. Registries within well-defined administrative boundaries will be included. Data will be collected from PubMed, Embase, Scopus, Web of Science, Lilacs, and references. Records will be independently screened by 2 reviewers, with any disagreements resolved through arbitration by a third reviewer. Homogeneous studies, with 3 or more evaluating the same outcome, may undergo meta-analysis. Subgroup analyses by registry type, injury groups, and other selected variables of interest will be conducted. Sensitivity analysis, risk of bias assessment, publication bias evaluation, and quality appraisal will also be performed., Results: This systematic review will run from November 2023 to June 2024. No identical review was found. Search strategies were finalized, the bibliographic search started, duplicates were eliminated, and title and abstract screening began. Of 35 studies retrieved, 85 were excluded due to duplication, leaving 50 for selection., Conclusions: This study is timely, aligning with ongoing national efforts to implement an injury registry. By synthesizing available evidence, we will identify the potential of injury registries to guide the decisions of Brazilian policy makers., Trial Registration: PROSPERO CRD42023481528; https://www.crd.york.ac.uk/prospero/display&#95;record.php?RecordID=481528., International Registered Report Identifier (irrid): PRR1-10.2196/55029., (©Ana Cláudia Medeiros-de-Souza, Luana Emanuelly Sinhori Lopes, Bruno Zocca de Oliveira, Edna Terezinha Rother, Lucas Reis Correia. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 30.10.2024.)

Background: (1) The "Systematic Review Support" system has a high degree of perceived usability., Background: (2)  Its use facilitates the exclusion of duplicate references and the selection of studies., Background: (3)  Remote and synchronous usability testing made research possible during the pandemic., Background: (4)  Combining usability techniques made it possible to identify the necessary adjustments., Background: (5)  The method used expanded the test's capacity in a real context of use., to evaluate the usability of the "Systematic Review Support" computer system., methodological study involving 21 students who evaluated the computer system in a remote and synchronous usability test. Two activities were carried out, containing 21 tasks simulating the initial steps of a review study, proposals for eliminating duplicate references, selecting studies and resolving disagreements between evaluators. At the end, participants answered the System Usability Scale questionnaire. The measures used were: completion rate with assistance (effectiveness) and time to complete the task (efficiency). The data was described in a table using the mean, standard deviation, minimum and maximum values., in a total of 21 tasks, the participants had an assisted completion rate of 9% and an average time of 55.1 minutes (the expected time being 60 minutes). The average score provided by the System Usability Scale instrument was 82.4 (considering a score from 0 to 100), median 85; minimum value 35 and maximum 97.5., the computer system has the effectiveness and efficiency parameters required of a computer review tool. It has a usability score of 82.4 on a scale of 0 to 100 assessed by the System Usability Scale instrument. The version presented incorporates the adjustments identified in the test in use.

Background: Systematic literature review (SLR), a robust method to identify and summarize evidence from published sources, is considered to be a complex, time-consuming, labor-intensive, and expensive task., Objective: This study aimed to present a solution based on natural language processing (NLP) that accelerates and streamlines the SLR process for observational studies using real-world data., Methods: We followed an agile software development and iterative software engineering methodology to build a customized intelligent end-to-end living NLP-assisted solution for observational SLR tasks. Multiple machine learning-based NLP algorithms were adopted to automate article screening and data element extraction processes. The NLP prediction results can be further reviewed and verified by domain experts, following the human-in-the-loop design. The system integrates explainable articificial intelligence to provide evidence for NLP algorithms and add transparency to extracted literature data elements. The system was developed based on 3 existing SLR projects of observational studies, including the epidemiology studies of human papillomavirus-associated diseases, the disease burden of pneumococcal diseases, and cost-effectiveness studies on pneumococcal vaccines., Results: Our Intelligent SLR Platform covers major SLR steps, including study protocol setting, literature retrieval, abstract screening, full-text screening, data element extraction from full-text articles, results summary, and data visualization. The NLP algorithms achieved accuracy scores of 0.86-0.90 on article screening tasks (framed as text classification tasks) and macroaverage F1 scores of 0.57-0.89 on data element extraction tasks (framed as named entity recognition tasks)., Conclusions: Cutting-edge NLP algorithms expedite SLR for observational studies, thus allowing scientists to have more time to focus on the quality of data and the synthesis of evidence in observational studies. Aligning the living SLR concept, the system has the potential to update literature data and enable scientists to easily stay current with the literature related to observational studies prospectively and continuously., (© Frank J Manion, Jingcheng Du, Dong Wang, Long He, Bin Lin, Jingqi Wang, Siwei Wang, David Eckels, Jan Cervenka, Peter C Fiduccia, Nicole Cossrow, Lixia Yao. Originally published in JMIR Medical Informatics (https://medinform.jmir.org).)

Introduction: Peritoneal dialysis (PD) is an effective home-based treatment for end-stage kidney disease (ESKD) that offers several advantages over in-centre haemodialysis, including better quality of life, increased autonomy and lower costs. Despite these benefits, PD remains underused globally, with wide variations in adoption rates across countries. Qualitative studies have explored the experiences, perceptions and decision-making processes of patients, caregivers and healthcare providers regarding PD, but their findings have not been systematically synthesised. This protocol outlines a systematic review of qualitative studies to identify and synthesise the multilevel factors that influence PD adoption., Methods and Analysis: We will conduct a comprehensive search of electronic databases (PubMed, Web of Science, Embase, CINAHL, MEDLINE, The Cochrane Library, PsycINFO, Scopus) and grey literature sources for qualitative studies published in English from each database inception to June 2024 that explore barriers and facilitators to PD adoption. Two reviewers will independently screen titles, abstracts and full texts for eligibility based on predefined criteria. Eligible studies will include those that use qualitative methods (eg, interviews, focus groups, observations) to explore the perspectives of adult ESKD patients, their caregivers and/or healthcare providers on factors influencing PD adoption, initiation or maintenance. Data will be extracted using a standardised form and synthesised using thematic analysis. The methodological quality of included studies will be appraised using the Critical Appraisal Skills Programme Qualitative Checklist. Confidence in the review findings will be assessed using the Grading of Recommendations Assessment, Development and Evaluation-Confidence in the Evidence from Reviews of Qualitative research approach., Ethics and Dissemination: Ethics approval is not required as this study will merely synthesise data from published studies. The results will be disseminated through peer-reviewed publications as well as conference presentations., Study Registration: PROSPERO, CRD42024570521., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Systematic review with meta-analysis integrates findings from multiple studies, offering robust conclusions on treatment effects and guiding evidence-based medicine. However, the process is often hampered by challenges such as inconsistent data reporting, complex calculations, and time constraints. Researchers must convert various statistical measures into a common format, which can be error-prone and labor-intensive without the right tools., Implementation: Meta-Analysis Accelerator was developed to address these challenges. The tool offers 21 different statistical conversions, including median & interquartile range (IQR) to mean & standard deviation (SD), standard error of the mean (SEM) to SD, and confidence interval (CI) to SD for one and two groups, among others. It is designed with an intuitive interface, ensuring that users can navigate the tool easily and perform conversions accurately and efficiently. The website structure includes a home page, conversion page, request a conversion feature, about page, articles page, and privacy policy page. This comprehensive design supports the tool's primary goal of simplifying the meta-analysis process., Results: Since its initial release in October 2023 as Meta Converter and subsequent renaming to Meta-Analysis Accelerator, the tool has gained widespread use globally. From March 2024 to May 2024, it received 12,236 visits from countries such as Egypt, France, Indonesia, and the USA, indicating its international appeal and utility. Approximately 46% of the visits were direct, reflecting its popularity and trust among users., Conclusions: Meta-Analysis Accelerator significantly enhances the efficiency and accuracy of meta-analysis of systematic reviews by providing a reliable platform for statistical data conversion. Its comprehensive variety of conversions, user-friendly interface, and continuous improvements make it an indispensable resource for researchers. The tool's ability to streamline data transformation ensures that researchers can focus more on data interpretation and less on manual calculations, thus advancing the quality and ease of conducting systematic reviews and meta-analyses., (© 2024. The Author(s).)

Introduction: The properties of creative products-novelty and usefulness-are generally agreed upon by researchers. Yet, consensus is lacking on which personal and environmental factors contribute to an individual's creative potential, or to what extent. substantial research work has been conducted in this area, leading to the publication of many systematic reviews and meta-analyses collating the available evidence. However, many of these reviews have differing methodological and theoretical characteristics, and often report conflicting results. To summarize the current review literature on factors associated with creativity and better understand the similarities and discrepancies among reviews on the same topic, we plan to conduct an umbrella review of reviews., Methods: This protocol has been registered in Open Science Framework (DOI: 10.17605/OSF.IO/H78YS). We will search MEDLINE, PsycINFO, Web of Science, EMBASE, and Google Scholar for peer-reviewed reviews exploring factors associated with creativity. The quality of the included reviews will be assessed using the Risk of Bias Assessment Tool for Systematic Reviews (ROBIS), and the degree of study overlap across reviews assessed through calculation of the corrected covered area (CCA). Descriptive analyses will be performed to synthesize the results of the included reviews. We plan to use the results to create a holistic framework of the factors associated with creativity, which could be used to cross-validate existing creativity theories, or create new ones., Ethics and Dissemination: The results of this umbrella review will be published in a relevant psychology-focused journal and presented at local, national and international conferences. As all data is peer-reviewed and presented in aggregate, we will not require ethics approval., Competing Interests: The authors have declared that no competing interests exist., (Copyright: © 2024 Thabane et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Background: The procedures used to assess the methodological quality and risk of bias (RoB) of systematic reviews of observational dental studies have not been investigated. The purpose of this research was to examine the way that authors of systematic reviews of epidemiological observational studies published in dentistry conducted the methodological assessment of those primary studies. In the present article, we aimed to assess the characteristics and the level of reporting of tools used to assess the methodologies of these reviews., Methods: We searched Scopus and the Web of Science from their inceptions to June 2023 for systematic reviews with meta-analyses of observational studies published in dentistry. Document selection and data extraction were performed in duplicate and independently by two authors. In a random sample of 10% of the systematic reviews, there was an agreement of more than 80% between the reviewers; data selection and extraction were conducted in the remaining 90% of the sample by one author. Data on the article and systematic review characteristics were extracted and recorded for descriptive reporting., Results: The search in the two databases resulted in the inclusion of 3,214 potential documents. After the elimination of duplicates and the application of the eligibility criteria, a total of 399 systematic reviews were identified and included. A total of 368 systematic reviews reported a methodological tool, of which 102 used the Newcastle-Ottawa scale. Additionally, 76 systematic reviews stated the use of a modified methodological tool. Information about the approach of assessing the methodological quality or RoB of primary studies but reporting no tool or tool name occurred in 25 reviews., Conclusions: The majority of authors of systematic reviews of epidemiological observational studies published in dentistry reported the tools used to assess the methodological quality or RoB of the included primary studies. Modifying existing tools to meet the individual characteristics of various studies should be considered., (© 2024. The Author(s).)

Objectives: Risk of bias (RoB) assessment is a critical part of any systematic review (SR). There are multiple tools available for assessing RoB of the studies included in a SR. The conduct of these assessments in intervention SRs are addressed by three items in AMSTAR-2, considered the preferred tool for critically appraising an intervention SR. This study focuses attention on item 9, which assesses the ability of a RoB tool to adequately address sources of bias, particularly in randomized trials (RCTs) and nonrandomized studies of interventions (NRSI). Our main objective is to report the detailed results of our examination of both Cochrane and non-Cochrane RoB tools and distinguish those that meet AMSTAR-2 item 9 appraisal standards., Study Design and Setting: We identified critical appraisal tools reported in a sample of 126 SRs reporting on interventions for persons with cerebral palsy published from 2014 to 2021. Eligible tools were those that had been used to assess the primary studies included in these SRs and for which assessment results were reported in enough detail to allow appraisal of the tool. We identified the version of the tool applied as original, modified, or novel and established the applicable study designs as intended by the tools' developers. We then evaluated the potential ability of these tools to assess the four sources of bias specified in AMSTAR-2 item 9 for RCTs and NRSI. We adapted item 9 to appraise tools applied to single-case experimental designs, which we also encountered in this sample of SRs., Results: Most of the eligible tools are recognized by name in the published literature and were applied in the original or modified form. Modifications were applied with considerable variability across the sample. Of the 37 tools we examined, those judged to fully meet the appraisal standards for RCTs included all the Cochrane tools, the original and modified Downs and Black Checklist, and the quality assessment standard for a cross-over study by Ding et al; for NRSI, these included all the Cochrane tools, the original and modified Downs and Black Checklist, and the Research Triangle Institute item bank on Risk of Bias and Precision of Observational Studies for NRSI. In general, tools developed for a specific study design were judged to meet the appraisal standards fully or partially for that design. These results suggest it is unlikely that a single tool will be adequate by AMSTAR-2 item 9 appraisal standards for an intervention SR that includes studies of various designs., Conclusion: To our knowledge, this is the first resource providing SR authors with practical information about the appropriateness and adequacy of RoB tools by the appraisal standards specified in AMSTAR-2 item 9 for RCTs and NRSI. We propose similar methods for appraisal of tools applied to single-case experimental design. We encourage authors to seek contemporary RoB tools developed for use in healthcare-related intervention SRs and designed to evaluate relevant study design features. The tools should address attributes unique to the review topic and research question but not be subjected to unjustified and excessive modifications. We promote recognition of the potential shortcomings of both Cochrane and non-Cochrane RoB tools, even those that perform well by AMSTAR-2 item 9 appraisal standards., Competing Interests: Declaration of competing interest None., (Copyright © 2024 Elsevier Inc. All rights reserved.)

Objectives: The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, first published in 2009, has been widely endorsed and compliance is high in systematic reviews (SRs) of intervention studies. SRs of prevalence studies are increasing in frequency, but their characteristics and reporting quality have not been examined in large studies. Our objectives were to describe the characteristics of SRs of prevalence studies in adults, evaluate the completeness of reporting, and explore study-level characteristics associated with the completeness of reporting., Study Design and Setting: We did a metaresearch study. We searched 5 databases from January 2010 to December 2020 to identify SRs of prevalence studies in adult populations. We used the PRISMA 2009 checklist to assess completeness of reporting and recorded additional characteristics. We conducted a descriptive analysis of review characteristics and linear regression to assess the relationship between compliance with PRISMA and publication characteristics., Results: We included 1172 SRs of prevalence studies. The number of reviews increased from 25 in 2010 to 273 in 2020. The median PRISMA score for SRs without meta-analysis was 17.5 of a maximum of 23, and for SRs with meta-analysis, 22 of a maximum of 25. Completeness of reporting, particularly for key items in the methods section, was suboptimal. SRs that included a meta-analysis or reported using a reporting or conduct guideline were the factors most strongly associated with increased compliance with PRISMA 2009., Conclusion: Reporting of SRs of prevalence was adequate for many PRISMA items. Nonetheless, this study highlights aspects for which special attention is needed. Development of a specific tool to assess the risk of bias in prevalence studies and an extension to the PRISMA statement could improve the conduct and reporting of SRs of prevalence studies., Competing Interests: Declaration of competing interest There are no competing interests., (Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: The US Agency for Healthcare Research and Quality, through the Evidence-based Practice Center (EPC) Program, aims to provide health system decision makers with the highest-quality evidence to inform clinical decisions. However, limitations in the literature may lead to inconclusive findings in EPC systematic reviews (SRs). The EPC Program conducted pilot projects to understand the feasibility, benefits, and challenges of utilizing health system data to augment SR findings to support confidence in healthcare decision-making based on real-world experiences., Study Design and Setting: Three contractors (each an EPC located at a different health system) selected a recently completed SR conducted by their center and identified an evidence gap that electronic health record (EHR) data might address. All pilot project topics addressed clinical questions as opposed to care delivery, care organization, or care disparities topics that are common in EPC reports. Topic areas addressed by each EPC included infantile epilepsy, migraine, and hip fracture. EPCs also tracked additional resources needed to conduct supplemental analyses. The workgroup met monthly in 2022-2023 to discuss challenges and lessons learned from the pilot projects., Results: Two supplemental data analyses filled an evidence gap identified in the SRs (raised certainty of evidence, improved applicability) and the third filled a health system knowledge gap. Project challenges fell under three themes: regulatory and logistical issues, data collection and analysis, and interpretation and presentation of findings. Limited ability to capture key clinical variables given inconsistent or missing data within the EHR was a major limitation. The workgroup found that conducting supplemental data analysis alongside an SR was feasible but adds considerable time and resources to the review process (estimated total hours to complete pilot projects ranged from 283 to 595 across EPCs), and that the increased effort and resources added limited incremental value., Conclusion: Supplementing existing SRs with analyses of EHR data is resource intensive and requires specialized skillsets throughout the process. While using EHR data for research has immense potential to generate real-world evidence and fill knowledge gaps, these data may not yet be ready for routine use alongside SRs., Competing Interests: Declaration of competing interest The authors have no relevant financial relationships to disclose., (Copyright © 2024. Published by Elsevier Inc.)

Objectives: Artificial intelligence (AI) tools utilizing machine learning (ML) have gained increasing utility in medicine and academia as a means of enhancing efficiency. ASReview is one such AI program designed to streamline the systematic review process through the automated prioritization of relevant articles for screening. This study examined the screening efficiency of ASReview when conducting systematic reviews and the potential factors that could influence its efficiency., Methods: Six distinct topics within the field of periodontics were searched in PubMed and Web of Science to obtain articles for screening within ASReview. Through a "training" process, relevant and irrelevant articles were manually incorporated to develop "prior knowledge" and facilitate ML optimization. Screening was then conducted following ASReview's algorithmically-generated relevance rankings. Screening efficiency was evaluated based on the normalized number of articles not requiring detailed review and on the total time expenditure., Results: Across the six topics, an average of 60.2 % of articles did not warrant extensive screening, given that all relevant articles were discovered within the first 39.8 % of publication reviewed. No significant variations in efficiencies were observed with differing methods of assembling prior knowledge articles or via modifications in article ratios and numbers., Conclusions: On average, ASReview conferred a 60.2 % improvement in screening efficiency, largely attributed to its dynamic ML capabilities. While advanced technologies like ASReview promise enhanced efficiencies, the accurate human discernment of article relevancy and quality remains indispensable when training these AI tools., Clinical Significance: Using ASReview has the potential to save approximately 60 % of time and effort required for screening articles., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

Objective: This study aims to evaluate the capacity of ChatGPT-4o to generate new systematic review ideas in the field of oral and maxillofacial surgery. The data obtained from this study will provide evidence-based information to oral and maxillofacial surgeons regarding the academic use of GPT-4o., Materials and Methods: ChatGPT-4o was asked to provide four previously unpublished systematic review ideas each for the topics of impacted third molars, dental implants, orthognathic surgery, and temporomandibular disorders. A literature search was conducted in the PubMed database to check if the ideas generated by GPT-4o had been previously published, and the search results were compared with the ideas generated by the AI., Results: The PubMed database search resulted in a total of 871 publications, with 37 publications found to be related to the topics generated by GPT-4o after the first and second screening. Out of the 16 publication ideas generated by GPT-4o, 9 (56.25 %) were determined to be previously unexplored according to the PubMed database search. There was no statistically significant relationship between the presence of ChatGPT's suggestions in PubMed and the subject areas of the studies., Conclusion: ChatGPT-4o has a high potential to be used as a valuable tool for suggesting systematic review topics in oral and maxillofacial surgery. Additionally, this tool can assist researchers not only in proposing publication ideas but also in developing the methodology of the study., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The author (Yunus Balel) is an Editorial Board Member for [Journal of Stomatology Oral and Maxillofacial Surgery] and was not involved in the editorial review or the decision to publish this article., (Copyright © 2024 Elsevier Masson SAS. All rights reserved.)

An assessment of the validity of studies is an essential component of most evidence syntheses (systematic reviews) to understand the risk of bias (ROB) and applicability of the evidence. A formal validity assessment requires a structured and comprehensive approach, which can be implemented using an assessment tool, specifically developed for this purpose. Many different tools are available, marking it difficult for researchers to choose the best tool for their evidence synthesis. We have established the LATITUDES Network to assist researchers in identifying the most appropriate tool to use in their evidence synthesis and to support researchers using these tools. The LATITUDES website (www.latitudes-network.org) includes a searchable library of validity assessment tools designed for use in evidence syntheses, bringing tools together in one place and providing researchers with clear information on suitable tools, categorized by study design. The website also provides links to training on the process of validity assessment and a list of tools currently under development. To be included in the LATITUDES library, tools must meet the following criteria: be designed for use in evidence syntheses; assess multidimensional aspects of validity of individual studies or reviews; and be developed for use by the wider research community rather than for a single research group. We highlight 'key' tools, those that are considered to be the most robust and reliable tools based on prespecified criteria agreed in conjunction with our advisory board, an international group of experts in the area of evidence synthesis and ROB tools., Competing Interests: Declaration of competing interest All authors are members of the LATITUDES Network leadership group. None of the authors have any additional competing interests to clear., (Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: To demonstrate how researchers can identify and translate reporting gaps from a systematic review into checklist items for reporting guidelines., Study Design and Setting: Good quality research reporting ensures transparency, reproducibility, and utility, facilitated by reporting guidelines. Conducting a systematic review is an essential step in the development of these guidelines. The Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network's toolkit (2010) assists researchers in this process and is due for an update to address current gaps and evolving research methods. One significant gap is the translation of systematic review findings into checklist items. Reflecting on our experience developing the ACcurate Consensus Reporting Document, we illustrate this translation process aiming to empower researchers developing reporting guidelines to address potential biases and promote transparency. We highlight the challenges faced and how they were addressed., Results: The systematic review search process was iterative, involving multiple adjustments to balance precision and sensitivity. Excessively stringent exclusion criteria may lead to missed valuable insights, especially when studies offer relevant content. An information specialist was invaluable in developing the search strategy. Key lessons learned include the necessity of maintaining flexibility and openness during data extraction, continuous adaptation based on panelist feedback, and promoting clear communication through understandable language. These principles can guide the development of future reporting guidelines and the updating of the EQUATOR toolkit, promoting transparency and robustness in research reporting., Conclusion: Maintaining flexibility, capturing evolving insights, clear communication, and accommodating changes in research and technologies are key to translating systematic review findings into effective reporting checklists., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this article., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Competing Interests: Competing interests: All coauthors have a direct or indirect interest in systematic reviews and replication as part of their job or academic career.

Background: Systematic reviews (SRs) are time-consuming and labor-intensive to perform. With the growing number of scientific publications, the SR development process becomes even more laborious. This is problematic because timely SR evidence is essential for decision-making in evidence-based healthcare and policymaking. Numerous methods and tools that accelerate SR development have recently emerged. To date, no scoping review has been conducted to provide a comprehensive summary of methods and ready-to-use tools to improve efficiency in SR production., Objective: To present an overview of primary studies that evaluated the use of ready-to-use applications of tools or review methods to improve efficiency in the review process., Methods: We conducted a scoping review. An information specialist performed a systematic literature search in four databases, supplemented with citation-based and grey literature searching. We included studies reporting the performance of methods and ready-to-use tools for improving efficiency when producing or updating a SR in the health field. We performed dual, independent title and abstract screening, full-text selection, and data extraction. The results were analyzed descriptively and presented narratively., Results: We included 103 studies: 51 studies reported on methods, 54 studies on tools, and 2 studies reported on both methods and tools to make SR production more efficient. A total of 72 studies evaluated the validity (n = 69) or usability (n = 3) of one method (n = 33) or tool (n = 39), and 31 studies performed comparative analyses of different methods (n = 15) or tools (n = 16). 20 studies conducted prospective evaluations in real-time workflows. Most studies evaluated methods or tools that aimed at screening titles and abstracts (n = 42) and literature searching (n = 24), while for other steps of the SR process, only a few studies were found. Regarding the outcomes included, most studies reported on validity outcomes (n = 84), while outcomes such as impact on results (n = 23), time-saving (n = 24), usability (n = 13), and cost-saving (n = 3) were less often evaluated., Conclusion: For title and abstract screening and literature searching, various evaluated methods and tools are available that aim at improving the efficiency of SR production. However, only few studies have addressed the influence of these methods and tools in real-world workflows. Few studies exist that evaluate methods or tools supporting the remaining tasks. Additionally, while validity outcomes are frequently reported, there is a lack of evaluation regarding other outcomes., (© 2024. The Author(s).)

Background: Systematic reviews and data synthesis of randomised clinical trials play a crucial role in clinical practice, research, and health policy. Trial sequential analysis can be used in systematic reviews to control type I and type II errors, but methodological errors including lack of protocols and transparency are cause for concern. We assessed the reporting of trial sequential analysis., Methods: We searched Medline and the Cochrane Database of Systematic Reviews from 1 January 2018 to 31 December 2021 for systematic reviews and meta-analysis reports that include a trial sequential analysis. Only studies with at least two randomised clinical trials analysed in a forest plot and a trial sequential analysis were included. Two independent investigators assessed the studies. We evaluated protocolisation, reporting, and interpretation of the analyses, including their effect on any GRADE evaluation of imprecision., Results: We included 270 systematic reviews and 274 meta-analysis reports and extracted data from 624 trial sequential analyses. Only 134/270 (50%) systematic reviews planned the trial sequential analysis in the protocol. For analyses on dichotomous outcomes, the proportion of events in the control group was missing in 181/439 (41%), relative risk reduction in 105/439 (24%), alpha in 30/439 (7%), beta in 128/439 (29%), and heterogeneity in 232/439 (53%). For analyses on continuous outcomes, the minimally relevant difference was missing in 125/185 (68%), variance (or standard deviation) in 144/185 (78%), alpha in 23/185 (12%), beta in 63/185 (34%), and heterogeneity in 105/185 (57%). Graphical illustration of the trial sequential analysis was present in 93% of the analyses, however, the Z-curve was wrongly displayed in 135/624 (22%) and 227/624 (36%) did not include futility boundaries. The overall transparency of all 624 analyses was very poor in 236 (38%) and poor in 173 (28%)., Conclusions: The majority of trial sequential analyses are not transparent when preparing or presenting the required parameters, partly due to missing or poorly conducted protocols. This hampers interpretation, reproducibility, and validity., Study Registration: PROSPERO CRD42021273811., (© 2024. The Author(s).)

Objectives: This is a protocol for a Cochrane Review (intervention). The objectives are as follows: To assess the effects of smoking cessation interventions on tobacco smoking in adults receiving inpatient psychiatry treatment. To assess whether the effects of smoking cessation interventions differ according to psychiatric diagnosis or type of intervention or comparator condition., (Copyright © 2024 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.)

Background: Electronic nicotine delivery systems (ENDS)-e-cigarettes or vapes-have been shown to substantially reduce or eliminate many toxins compared with cigarette smoke, but simultaneously ENDS use also produces their own unique toxins. Yet the patterns of use among people who use ENDS are not homogeneous. Some people who use ENDS also smoke cigarettes (dual use). Other people who formerly smoked cigarettes are completely substituting ENDS (exclusive use). A small number of people who have never smoked cigarettes are using ENDS (naïve use of nicotine). Each of these patterns of use results in different exposures to toxins. Unfortunately, epidemiological studies routinely group together any ENDS use regardless of other tobacco use., Objective: This umbrella review primarily aims to present all the evidence available on the respiratory effects of ENDS use by adults based on their pattern of use: dual use, exclusive use, and naïve use. With each of these patterns of use, are there benefits, no changes, or harmful effects on respiratory functioning? Our objective is to provide clinicians with a detailed analysis of how different patterns of ENDS use impact respiratory functioning and to point to the best sources of evidence., Methods: This umbrella review follows the Methods for Overviews of Reviews framework and the PRIOR (Preferred Reporting Items for Overviews of Reviews) statement. Systematic reviews published since 2019 will be searched across 4 databases and 3 gray literature sources. Additional searches will include citation chasing, references lists, and referrals from respiratory specialists. The quality of included reviews will be evaluated using the AMSTAR2 (A Measurement Tool to Assess Systematic Reviews) checklist. We will document biases in 3 areas: protocol deviations, biases from the Oxford Catalogue of Bias, and internal data discrepancies. Two reviewers will independently conduct the search and quality assessments. Our analysis will focus on reviews rated as moderate or high confidence by AMSTAR2. We will use the Vote Counting Direction of Effect method to manage expected data heterogeneity, assessing whether ENDS use is beneficial or detrimental, or has no effect on respiratory functions based on the pattern of use., Results: The review is expected to be completed by December 2024. The database search was concluded in April 2024, and data extraction and bias assessment were completed in June 2024. The analysis phase is planned to be completed by October 2024., Conclusions: A thorough and comprehensive assessment of the evidence will better inform the contentious debate over the respiratory effects of ENDS providing much needed clarity by linking their effects to specific usage patterns. This analysis is particularly crucial in understanding the risks associated with continued cigarette smoking., Trial Registration: PROSPERO CRD42024540034; https://www.crd.york.ac.uk/prospero/display&#95;record.php?RecordID=540034., International Registered Report Identifier (irrid): DERR1-10.2196/60325., (©Giusy Rita Maria La Rosa, Riccardo Polosa, Renée O’Leary. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 04.09.2024.)

Background: Generative artificial intelligence (GenAI) shows promise in automating key tasks involved in conducting systematic literature reviews (SLRs), including screening, bias assessment and data extraction. This potential automation is increasingly relevant as pharmaceutical developers face challenging requirements for timely and precise SLRs using the population, intervention, comparator and outcome (PICO) framework, such as those under the impending European Union (EU) Health Technology Assessment Regulation 2021/2282 (HTAR). This proof-of-concept study aimed to evaluate the feasibility, accuracy and efficiency of using GenAI for mass extraction of PICOs from PubMed abstracts., Methods: Abstracts were retrieved from PubMed using a search string targeting randomised controlled trials. A PubMed clinical study 'specific/narrow' filter was also applied. Retrieved abstracts were processed using the OpenAI Batch application programming interface (API), which allowed parallel processing and interaction with Generative Pre-trained Transformer 4 Omni (GPT-4o) via custom Python scripts. PICO elements were extracted using a zero-shot prompting strategy. Results were stored in CSV files and subsequently imported into a PostgreSQL database., Results: The PubMed search returned 682,667 abstracts. PICOs from all abstracts were extracted in < 3 h, with an average processing time of 200 s per 1000 abstracts. A total of 395,992,770 tokens were processed, with an average of 580 tokens per abstract. The total cost was $3390. On the basis of a random sample of 350 abstracts, human verification confirmed that GPT-4o accurately and comprehensively extracted 342 (98%) of all PICOs, with only outcome elements rarely missed., Conclusions: Using GenAI to extract PICOs from clinical study abstracts could fundamentally transform the way SLRs are conducted. By enabling pharmaceutical developers to anticipate PICO requirements, this approach allows for proactive preparation for the EU HTAR process, or other health technology assessments (HTAs), streamlining efficiency and reducing the burden of meeting these requirements., (© 2024. The Author(s).)

Introduction: There is growing interest in the use of ChatGPT in the writing and reviewing of scientific articles. In line with the nature of ChatGPT, we tested its effectiveness in the scientific article review process., Methods: We compared the findings of a systematic review of the published literature, produced by researchers in the traditional way, with a version created by ChatGPT, obtained by providing the same inputs as the original paper and a set of instructions (prompts) optimized to obtain the same type of result; we also identified the process that led to a comparable result. In order to assess the effectiveness of ChatGPT in analyzing the systematic review, we selected an existing, replicable study on the experience of health care professionals in the use of digital tools in clinical practice, from which we extracted and downloaded the related 17 publications in Pdf format. Subsequently, we uploaded these references into ChatGPT, setting specific prompts detailing the professional profile required, the context of the application, the expected outputs, and the level of creative freedom (temperature) to a minimum to limit the possibility of "hallucinations". After verifying ChatGPT's understanding of the task, we performed several iterations of the prompt until we obtained a result comparable to the original review. Finally, we systematically compared the results obtained by ChatGPT with those of the reference review., Results: The analysis showed that ChatGPT's results are comparable to human results, although 4 iterations of the prompt are required to approach the human benchmark., Discussion: Although ChatGPT showed comparable capabilities in text review, human authors exhibited greater analytical depth in interpretation. Due to their greater creative freedom, the authors offered more details about the benefits of digital tools in the hospital setting. ChatGPT, however, enriched the analysis by including elements not contemplated initially. The final comparison revealed comparable macro-themes between the two approaches, emphasizing the need for careful human validation to ensure the full integrity and depth of the analysis., Conclusions: Generative artificial intelligence (AI), represented by ChatGPT, showed significant potential in revolutionizing the production of scientific literature by supporting healthcare professionals. Although there are challenges that require careful evaluation, ChatGPT's results are comparable to human results. The key element is not so much the superiority of AI over humans but the human ability to configure and direct AI for optimal or even potentially superior human results.

The author describes various ways to assess the quality of systematic reviews, particularly meta-analyses., Competing Interests: Conflict of Interest The author reports no conflicts of interest or relevant financial relationships., (Copyright © 2024. Published by Elsevier Inc.)

Objectives: The aim of this paper is to provide clinicians and authors of clinical guidelines or patient information with practical guidance on searching and choosing systematic reviews(s) (SR[s]) and, where adequate, on making use of SR(s)., Study Design and Setting: At the German conference of the Evidence-Based Medicine Network (EbM Network) a workshop on the topic was held to identify the most important areas where guidance for practice appears necessary. After the workshop, we established working groups. These included SR users with different backgrounds (eg, information specialists, epidemiologists) and working areas. Each working group developed and consented a draft guidance based on their expert knowledge and experiences. The results were presented to the entire group and finalized in an iterative process., Results: We developed a practical guidance that answers questions that usually arise when choosing and using SR(s). (1) How to efficiently find high-quality SRs? (2) How to choose the most appropriate SR? (3) What to do if no SR of sufficient quality could be identified? In addition, we developed an algorithm that links these steps and accounts for their interaction. The resulting guidance is primarily directed at clinicians and developers of clinical practice guidelines or patient information resources., Conclusion: We suggest practical guidance for making the best use of SRs when answering a specific research question. The guidance may contribute to the efficient use of existing SRs. Potential benefits when using existing SRs should be always weighted against potential limitations., Competing Interests: Declaration of competing interest There are no competing interests for any author., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objectives: To map whether and how systematic reviews (SRs) with network meta-analysis (NMA) use presentation formats to report (a) structured evidence summaries - here defined as reporting of effects estimates in absolute effects with certainty ratings and with a method to rate interventions across one or more outcome(s) - and (b) NMA results in general., Study Design and Setting: We conducted a systematic survey, searching MEDLINE (Ovid) for SRs with NMA published between January 1, 2020, and December 31, 2021. We planned to include a random sample of publications, with predefined mechanisms in place for saturation, and included SRs that met prespecified quality criteria and extracted data on presentation formats that reported: (a) estimates of effects, (b) certainty of the evidence, or (c) rating of interventions., Results: The 200 eligible SRs, from 158 unique Journals, utilized 1133 presentation formats. We found structured evidence summaries in 10 publications (5.0%), with 3 (1.5%) reporting structured evidence summaries across all outcomes, including benefits and harms. Sixteen of the 133 SRs (11.7%) reporting dichotomous outcomes included estimates of absolute effects. Seventy-six SRs (38.0%) reported both benefits and harms and 26 SRs (13.0%) reported certainty ratings in presentation formats, 20 (76.9%) used Grading of Recommendations Assessment, Development and Evaluation and 6 (23.1%) used Confidence In Network Meta-analysis. Surface Under the Cumulative Ranking Curve was the most common method to rate interventions (69 SRs, 34.5%). NMA results were most often reported using forest plots (108 SRs, 54.0%) and league tables (93 SRs, 46.5%)., Conclusion: Most SRs with NMA do not report structured evidence summaries and only rarely do such summaries include reporting of both benefits and harms; those that do offer effective user-friendly communication and provide models for optimal NMA presentation practice., Competing Interests: Declaration of competing interest T.A. reports a relationship with MAGIC Evidence Ecosystem Foundation that includes: board membership and employment. P.O.V. reports a relationship with MAGIC Evidence Ecosystem Foundation that includes: board membership and employment. P.O.V. and T.A. are CEO and deputy CEO of the nonprofit organization MAGIC Evidence Ecosystem Foundation (https://www.magicevidence.org). In collaboration with P.O.L. at Lovisenberg Diaconal Hospital, MAGIC is conducting the research project Making Alternative Treatment Choices Intuitive and Trustworthy (MATCH-IT). The overarching aim of MATCH-IT is to develop a new interactive SoF table and decision support tool, displaying multiple structured evidence summaries from NMAs. The tool may be integrated as part of MAGICapp (https://app.magicapp.org/#/guidelines), an online author and publication platform for guidelines, evidence summaries and decision aids. A.A.V. is Statistical Associate Editor for the Journal of Clinical Epidemiology, but had no involvement with the peer review process or decision for publication. A.C.T. is Coeditor-in-Chief for the Journal of Clinical Epidemiology, but had no involvement with the peer review process or decision for publication. X.W., T.P., M.Ø., and C.L. declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.)

Background: Developmental and Epileptic Encephalopathies (DEEs) are defined by drug-resistant seizures and neurodevelopmental disorders. Over 50% of patients have a genetic cause. Studies have shown that patients with DEEs, regardless of genetic diagnosis, experience a central visual function disorder known as Cerebral (cortical) Visual Impairment (CVI). The prevalence of CVI in DEE patients is currently unknown. A quantitative synthesis of existing data on the prevalence rates of this condition would aid in understanding the magnitude of the problem, outlining future research, and suggesting the need for therapeutic strategies for early identification and prevention of the disorder., Methods: The protocol followed the PRISMA-P statement for systematic review and meta-analysis protocols. The review will adhere to the JBI Manual for Evidence Synthesis (Systematic Reviews of Prevalence and Incidence) and use the CoCoPop framework to establish eligibility criteria. We will conduct a comprehensive search of several databases, including MEDLINE, EMBASE, Science Direct, Scopus, PsychINFO, Wiley, Highwire Press, and Cochrane Library of Systematic Reviews. Our primary focus will be determining the prevalence of cerebral visual impairments (Condition) in patients with developmental and epileptic encephalopathy (Population). To ensure clarity, we will provide a narrative summary of the risk of bias in the studies we include. The Cochrane Q statistic will be used to assess heterogeneity between studies. If the quantitative synthesis includes more than 10 studies, potential sources of heterogeneity will be investigated through subgroup and meta-regression analyses. Meta(bias)es analysis will also be performed. The quality of evidence for all outcomes will be evaluated using the Grading of Recommendations Assessment Development and Evaluation (GRADE) working group methodology., Discussion: This protocol outlines a systematic review and meta-analysis to identify, collect, evaluate, and integrate epidemiological knowledge related to the prevalence of CVI in patients with DEEs. To the best of our knowledge, no other systematic review and meta-analysis has addressed this specific issue. The results will provide useful information for understanding the extent of the problem, outlining future research, and suggesting the need for early identification strategies., Systematic Review Registrations: This Systematic Review Protocol was registered in PROSPERO (CRD42023448910)., (© 2024. The Author(s).)

Introduction: Systematic Review and Meta-Analysis (SRMAs) in neurosurgery have significantly increased. With approximately 1 million patients affected by cerebrovascular disease annually, interpreting SRMAs necessitates a systematic approach. The objective of this review is to identify and describe four essential domains for SRMA interpretation., Methods: This review outlines the necessities of reviewing existing literature and methodological frameworks essential for interpreting cerebrovascular neurosurgery SRMAs. Each domain is to accurately assess study design variations, heterogeneity assessment methods, outcome comparability strategies, and the impact of technological advancements and time bias on study outcomes., Results: Study design evaluation distinguishes between randomized controlled trials (RCTs) and non-randomized studies. RCTs provide high internal validity, but as seen in the ARUBA trial, can contain internal flaws that necessitate a deeper understanding before application to clinical practices. Non-randomized studies offer valuable real-world insights. A heterogeneity assessment involves readers and writers accurately using forest plots, Cochrane's Q test, Higgins I² statistics, subgroup analysis, and meta-regressions to understand a study's clinical findings. The expertise thresholds, as in the NASCET trial, significantly impact a study's external validity. Strategies such as the GRADE approach can assist in managing diverse outcome measures. Technological advancements, particularly in endovascular procedures and SRS, influence the accuracy of comparing studies across periods, and thus swiftly outdate older studies, lowering the applicability of SRMAs., Conclusion: Effective interpretation of cerebrovascular neurosurgery SRMAs requires attention to study design, heterogeneity, outcome comparability, and technological advancements. These domains collectively enable evidence-based clinical decision-making and optimized patient care in a dynamic field., (© 2024. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)

Background: To describe the algorithm and investigate the efficacy of a novel systematic review automation tool "the Deduplicator" to remove duplicate records from a multi-database systematic review search., Methods: We constructed and tested the efficacy of the Deduplicator tool by using 10 previous Cochrane systematic review search results to compare the Deduplicator's 'balanced' algorithm to a semi-manual EndNote method. Two researchers each performed deduplication on the 10 libraries of search results. For five of those libraries, one researcher used the Deduplicator, while the other performed semi-manual deduplication with EndNote. They then switched methods for the remaining five libraries. In addition to this analysis, comparison between the three different Deduplicator algorithms ('balanced', 'focused' and 'relaxed') was performed on two datasets of previously deduplicated search results., Results: Before deduplication, the mean library size for the 10 systematic reviews was 1962 records. When using the Deduplicator, the mean time to deduplicate was 5 min per 1000 records compared to 15 min with EndNote. The mean error rate with Deduplicator was 1.8 errors per 1000 records in comparison to 3.1 with EndNote. Evaluation of the different Deduplicator algorithms found that the 'balanced' algorithm had the highest mean F1 score of 0.9647. The 'focused' algorithm had the highest mean accuracy of 0.9798 and the highest recall of 0.9757. The 'relaxed' algorithm had the highest mean precision of 0.9896., Conclusions: This demonstrates that using the Deduplicator for duplicate record detection reduces the time taken to deduplicate, while maintaining or improving accuracy compared to using a semi-manual EndNote method. However, further research should be performed comparing more deduplication methods to establish relative performance of the Deduplicator against other deduplication methods., (© 2024. The Author(s).)

Background: Standard systematic review can be labor-intensive and time-consuming meaning that it can be difficult to provide timely evidence when there is an urgent public health emergency such as a pandemic. The ClinicalTrials.gov provides a promising way to accelerate evidence production., Methods: We conducted a search on PubMed to gather systematic reviews containing a minimum of 5 studies focused on safety aspects derived from randomized controlled trials (RCTs) of pharmacological interventions, aiming to establish a real-world dataset. The registration information of each trial from eligible reviews was further collected and verified. The meta-analytic data were then re-analyzed by using 1) the full meta-analytic data with all trials and 2) emulated rapid data with trials that had been registered and posted results on ClinicalTrials.gov, under the same synthesis methods. The effect estimates of the full meta-analysis and rapid meta-analysis were then compared., Results: The real-world dataset comprises 558 meta-analyses. Among them, 56 (10.0%) meta-analyses included RCTs that were not registered in ClinicalTrials.gov. For the remaining 502 meta-analyses, the median percentage of RCTs registered within each meta-analysis is 70.1% (interquartile range: 33.3% to 88.9%). Under a 20% bias threshold, rapid meta-analyses conducted through ClinicalTrials.gov achieved accurate point estimates ranging from 77.4% (using the MH model) to 83.1% (using the GLMM model); 91.0% to 95.3% of these analyses accurately predicted the direction of effects., Conclusions: Utilizing the ClinicalTrials.gov platform for safety assessment with a minimum of 5 RCTs holds significant potential for accelerating evidence synthesis to support urgent decision-making., (© 2024. The Author(s).)

Background: High-quality cardiopulmonary resuscitation (CPR) can restore spontaneous circulation (ROSC) and neurological function and save lives. We conducted an umbrella review, including previously published systematic reviews (SRs), that compared mechanical and manual CPR; after that, we performed a new SR of the original studies that were not included after the last published SR to provide a panoramic view of the existing evidence on the effectiveness of CPR methods., Methods: PubMed, EMBASE, and Medline were searched, including English in-hospital (IHCA) and out-of-hospital cardiac arrest (OHCA) SRs, and comparing mechanical versus manual CPR. A Measurement Tool to Assess Systematic Reviews (AMSTAR-2) and GRADE were used to assess the quality of included SRs/studies. We included both IHCA and OHCA, which compared mechanical and manual CPR. We analyzed at least one of the outcomes of interest, including ROSC, survival to hospital admission, survival to hospital discharge, 30-day survival, and survival to hospital discharge with good neurological function. Furthermore, subgroup analyses were performed for age, gender, initial rhythm, arrest location, and type of CPR devices., Results: We identified 249 potentially relevant records, of which 238 were excluded. Eleven SRs were analyzed in the Umbrella review (January 2014-March 2022). Furthermore, for a new, additional SR, we identified eight eligible studies (not included in any prior SR) for an in-depth analysis between April 1, 2021, and February 15, 2024. The higher chances of using mechanical CPR for male patients were significantly observed in three studies. Two studies showed that younger patients received more mechanical treatment than older patients. However, studies did not comment on the outcomes based on the patient's gender or age. Most SRs and studies were of low to moderate quality. The pooled findings did not show the superiority of mechanical compared to manual CPR except in a few selected subgroups., Conclusions: Given the significant heterogeneity and methodological limitations of the included studies and SRs, our findings do not provide definitive evidence to support the superiority of mechanical CPR over manual CPR. However, mechanical CPR can serve better where high-quality manual CPR cannot be performed in selected situations., (© 2024. The Author(s).)

Systematic reviews and meta-analyses are essential tools in contemporary evidence-based medicine, synthesizing evidence from various sources to better inform clinical decision-making. However, the conclusions from different meta-analyses on the same topic can be discrepant, which has raised concerns about their reliability. One reason is that the result of a meta-analysis is sensitive to factors such as study inclusion/exclusion criteria and model assumptions. The arm-based meta-analysis model is growing in importance due to its advantage of including single-arm studies and historical controls with estimation efficiency and its flexibility in drawing conclusions with both marginal and conditional effect measures. Despite its benefits, the inference may heavily depend on the heterogeneity parameters that reflect design and model assumptions. This article aims to evaluate the robustness of meta-analyses using the arm-based model within a Bayesian framework. Specifically, we develop a tipping point analysis of the between-arm correlation parameter to assess the robustness of meta-analysis results. Additionally, we introduce some visualization tools to intuitively display its impact on meta-analysis results. We demonstrate the application of these tools in three real-world meta-analyses, one of which includes single-arm studies., (© 2024. The Author(s).)

This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RRs) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. This paper guides how to use supportive software for RRs.We strongly encourage the use of supportive software throughout RR production. Specifically, we recommend (1) using collaborative online platforms that enable working in parallel, allow for real-time project management and centralise review details; (2) using automation software to support, but not entirely replace a human reviewer and human judgement and (3) being transparent in reporting the methodology and potential risk for bias due to the use of supportive software., Competing Interests: Competing interests: LA is an associate convenor of the RRMG. BNS, CH and GG are co-convenors of the RRMG., (© Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Systematic reviews (SRs) are being published at an accelerated rate. Decision-makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision-makers (eg, practitioners, policymakers, researchers) use SRs to inform decision-making and to explore the potential role of a proposed artificial intelligence (AI) tool to assist in critical appraisal and choosing among SRs., Methods: We developed a survey with 21 open and closed questions. We followed a knowledge translation plan to disseminate the survey through social media and professional networks., Results: Our survey response rate was lower than expected (7.9% of distributed emails). Of the 684 respondents, 58.2% identified as researchers, 37.1% as practitioners, 19.2% as students and 13.5% as policymakers. Respondents frequently sought out SRs (97.1%) as a source of evidence to inform decision-making. They frequently (97.9%) found more than one SR on a given topic of interest to them. Just over half (50.8%) struggled to choose the most trustworthy SR among multiple. These difficulties related to lack of time (55.2%), or difficulties comparing due to varying methodological quality of SRs (54.2%), differences in results and conclusions (49.7%) or variation in the included studies (44.6%). Respondents compared SRs based on the relevance to their question of interest, methodological quality, and recency of the SR search. Most respondents (87.0%) were interested in an AI tool to help appraise and compare SRs., Conclusions: Given the identified barriers of using SR evidence, an AI tool to facilitate comparison of the relevance of SRs, the search and methodological quality, could help users efficiently choose among SRs and make healthcare decisions., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Involving librarians as team members can lead to better quality in reviews. To improve their search results, an international diabetes project involved two medical librarians in a large-scale project planning of a series of systematic reviews for clinical guidelines in diabetes precision medicine., Case Presentation: The precision diabetes project was divided into teams. Four diabetes mellitus types (type 1, type 2, gestational, and monogenic) were divided into teams focusing on diagnostics, prevention, treatment, or prognostics. A search consultation plan was set up for the project to help organize the work. We performed searches in Embase and PubMed for 14 teams, building complex searches that involved non-traditional search strategies. Our search strategies generated very large amounts of records that created challenges in balancing sensitivity with precision. We also performed overlap searches for type 1 and type 2 diabetes search strategies; and assisted in setting up reviews in the Covidence tool for screening., Conclusions: This project gave us opportunities to test methods we had not used before, such as overlap comparisons between whole search strategies. It also gave us insights into the complexity of performing a search balancing sensitivity and specificity and highlights the need for a clearly defined communication plan for extensive evidence synthesis projects., (Copyright © 2024 Maria Björklund, Krister Aronsson.)

Objective: To determine if librarian collaboration was associated with improved database search quality, search reproducibility, and systematic review reporting in otolaryngology systematic reviews and meta-analyses., Methods: In this retrospective cross-sectional study, PubMed was queried for systematic reviews and meta-analyses published in otolaryngology journals in 2010, 2015, and 2021. Two researchers independently extracted data. Two librarians independently rated search strategy reproducibility and quality for each article. The main outcomes include association of librarian involvement with study reporting quality, search quality, and publication metrics in otolaryngology systematic reviews and meta-analyses. Categorical data were compared with Chi-Squared tests or Fisher's Exact tests. Continuous variables were compared via Mann Whitney U Tests for two groups, and Kruskal-Wallis Tests for three or more groups., Results: Of 559 articles retrieved, 505 were analyzed. More studies indicated librarian involvement in 2021 (n=72, 20.7%) compared to 2015 (n=14, 10.4%) and 2010 (n=2, 9.0%) (p=0.04). 2021 studies showed improvements in properly using a reporting tool (p<0.001), number of databases queried (p<0.001), describing date of database searches (p<0.001), and including a flow diagram (p<0.001). Librarian involvement was associated with using reporting tools (p<0.001), increased number of databases queried (p<0.001), describing date of database search (p=0.002), mentioning search peer reviewer (p=0.02), and reproducibility of search strategies (p<0.001). For search strategy quality, librarian involvement was associated with greater use of "Boolean & proximity operators" (p=0.004), "subject headings" (p<0.001), "text word searching" (p<0.001), and "spelling/syntax/line numbers" (p<0.001). Studies with librarian involvement were associated with publication in journals with higher impact factors for 2015 (p=0.003) and 2021 (p<0.001)., Conclusion: Librarian involvement was associated with improved reporting quality and search strategy quality. Our study supports the inclusion of librarians in review teams, and journal editing and peer reviewing teams., Competing Interests: The authors declare that there is no conflict of interest., (Copyright © 2024 Rachel Whitney, Michael C. Shih, Tamar Gordis, Shaun A. Nguyen, Ted A. Meyer, Emily A. Brennan.)

The increasing adoption of real-world studies in healthcare for decision making and planning has further necessitated the need for a specific quality assessment tool for evidence synthesis. This study aimed to develop a quality assessment tool for systematic reviews (SR) and meta-analysis (MA) involving real-world studies (QATSM-RWS) using a formal consensus method. Based on scoping review, the authors identified a list of items for possible inclusion in the quality assessment tool. A Delphi survey was formulated based on the identified items. A total of 89 experts, purposively recruited, with research experience in real-world data were invited to participate in the first round of Delphi survey. The participants who responded in the first Delphi round were invited to participate (n = 15) in the phrasing of the items. Strong level of agreement was found on the proposed list of items after the first round of Delphi. A rate of agreement ≥ 0.70 was used to define which items to keep in the tool. A list of 14 items emerged as suitable for QATSM-RWS. The items were structured under five domains: introduction, methods, results, discussions, and others. All participants agreed with the proposed phrasing of the items. This is the first study that has developed a specific tool that can be used to appraise the quality of SR and MA involving real-world studies. QATSM-RWS may be used by policymakers, clinicians, and practitioners when evaluating and generating real-world evidence. This tool is now undergoing validation process., (© 2024. The Author(s).)

In recent years, as a result of the dramatic increase in the number of systematic reviews, a new type of systematic review, the 'systematic reviews of systematic reviews', also known as umbrella reviews, reviews of reviews, meta-reviews or synthesis of review, was developed. The aim of this article is to provide recommendations on how this type of systematic review should be conducted and reported to ensure its quality and usefulness. These reviews are designed to compile evidence from multiple systematic reviews of interventions into an accessible and usable document and are one of the highest levels of evidence synthesis.

Don't let fear get in the way of greatness., (Copyright © 2024 Wolters Kluwer Health, Inc. All rights reserved.)

Objectives: To assess to what extent the overall quality of evidence indicates changes to observe intervention effect estimates when new data become available., Methods: We conducted a meta-epidemiological study. We obtained evidence from meta-analyses of randomized trials of Cochrane reviews addressing the same health-care question that was updated with inclusion of additional data between January 2016 and May 2021. We extracted the reported effect estimates with 95% confidence intervals (CIs) from meta-analyses and corresponding GRADE (Grading of Recommendations Assessment, Development, and Evaluation) assessments of any intervention comparison for the primary outcome in the first and the last updated review version. We considered the reported overall quality (certainty) of evidence (CoE) and specific evidence limitations (no, serious or very serious for risk of bias, imprecision, inconsistency, and/or indirectness). We assessed the change in pooled effect estimates between the original and updated evidence using the ratio of odds ratio (ROR), absolute ratio of odds ratio (aROR), ratio of standard errors (RoSE), direction of effects, and level of statistical significance., Results: High CoE without limitations characterized 19.3% (n = 29) out of 150 included original Cochrane reviews. The update with additional data did not systematically change the effect estimates (mean ROR 1.00; 95% CI 0.99-1.02), which deviated 1.06-fold from the older estimates (median aROR; interquartile range [IQR]: 1.01-1.15), gained precision (median RoSE 0.87; IQR 0.76-1.00), and maintained the same direction with the same level of statistical significance in 93% (27 of 29) of cases. Lower CoE with limitations characterized 121 original reviews and graded as moderate CoE in 30.0% (45 of 150), low CoE in 32.0% (48 of 150), and very low CoE in 18.7% (28 of 150) reviews. Their update had larger absolute deviations (median aROR 1.12 to 1.33) and larger gains in precision (median RoSE 0.78-0.86) without clear and consistent differences between these categories of CoE. Changes in effect direction or statistical significance were also more common in the lower quality evidence, again with a similar extent across categories (without change in 75.6%, 64.6%, and 75.0% for moderate, low, very low CoE). As limitations increased, effect estimates deviated more (aROR 1.05 with zero, 1.11 with one, 1.25 with two, 1.24 with three limitations) and changes in direction or significance became more frequent (93.2% stable with no limitations, 74.5% with one, 68.2% with two, and 61.5% with three limitations)., Conclusion: High-quality evidence without methodological deficiencies is trustworthy and stable, providing reliable intervention effect estimates when updated with new data. Evidence of moderate and lower quality may be equally prone to being unstable and cannot indicate if available effect estimates are true, exaggerated, or underestimated., Competing Interests: Declaration of competing interest I.H.: has one-year consulting contract for research projects at the Medical University of South Carolina, Charleston, SC, USA. D.N.: declares having led/coauthored/coauthoring Cochrane Reviews and conducted GRADE assessments as part of completing some of these reviews. He also declares having been part of the Cochrane Convenes initiative organized by Cochrane to consider the issue of misinformation, its impact on the health evidence ecosystem and solutions to address it. L.G.H.’s Institution (RC2NB) is supported by Foundation Clinical Neuroimmunology and Neuroscience Basel. There are no competing interests for any other author., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Introduction: Systematic reviews (SRs) and meta-analyses (MAs) are methods of data analysis used to synthesize information presented in multiple publications on the same topic. A thorough understanding of the steps involved in conducting this type of research and approaches to data analysis is critical for appropriate understanding, interpretation, and application of the findings of these reviews., Methods: We reviewed reference texts in clinical neuroepidemiology, neurostatistics and research methods and other previously related articles on meta-analyses (MAs) in surgery. Based on existing theories and models and our cumulative years of expertise in conducting MAs, we have synthesized and presented a detailed pragmatic approach to interpreting MAs in Neurosurgery., Results: Herein we have briefly defined SRs sand MAs and related terminologies, succinctly outlined the essential steps to conduct and critically appraise SRs and MAs. A practical approach to interpreting MAs for neurosurgeons is described in details. Based on summary outcome measures, we have used hypothetical examples to illustrate the Interpretation of the three commonest types of MAs in neurosurgery: MAs of Binary Outcome Measures (Pairwise MAs), MAs of proportions and MAs of Continuous Variables. Furthermore, we have elucidated on the concepts of heterogeneity, modeling, certainty, and bias essential for the robust and transparent interpretation of MAs. The basics for the Interpretation of Forest plots, the preferred graphical display of data in MAs are summarized. Additionally, a condensation of the assessment of the overall quality of methodology and reporting of MA and the applicability of evidence to patient care is presented., Conclusion: There is a paucity of pragmatic guides to appraise MAs for surgeons who are non-statisticians. This article serves as a detailed guide for the interpretation of systematic reviews and meta-analyses with examples of applications for clinical neurosurgeons., (© 2024. The Author(s), under exclusive licence to Springer-Verlag GmbH Austria, part of Springer Nature.)

In Part I of this series, we provide guidance for preparing a systematic review protocol. In this article, we highlight important steps and supplement with exemplars on conducting and reporting the results of a systematic review. We suggest how authors can manage protocol violations, multiplicity of outcomes and analyses, and heterogeneity. The quality (certainty) of the evidence and strength of recommendations should follow the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. It is our goal that Part II of this series provides valid guidance to authors and peer reviewers who conduct systematic reviews to adhere to important constructs of transparency, structure, reproducibility, and accountability. This will likely result in more rigorous systematic reviews being submitted for publication to the journals like Regional Anesthesia & Pain Medicine and Anesthesia & Analgesia ., Competing Interests: Competing interests: RSD’S is an Associate Editor for Anesthesia & Analgesia, and an Associate Editor for Regional Anesthesia & Pain Medicine. MJB is a Regional Anesthesia and Acute Pain Medicine Executive Section Editor of Anesthesia & Analgesia. AS received a honorarium for his contribution to this manuscript. EJM is Statistical Editor for Anesthesia & Analgesia. GAK is the Statistical Advisor for Systematic Reviews and Meta-analyses for Regional Anesthesia & Pain Medicine., (© American Society of Regional Anesthesia & Pain Medicine 2024. No commercial re-use. See rights and permissions. Published by BMJ.)