Search

Abstract Background As evidence related to the COVID-19 pandemic surged, databases, platforms, and repositories evolved with features and functions to assist users in promptly finding the most relevant evidence. In response, research synthesis teams adopted novel searching strategies to sift through the vast amount of evidence to synthesize and disseminate the most up-to-date evidence. This paper explores the key database features that facilitated systematic searching for rapid evidence synthesis during the COVID-19 pandemic to inform knowledge management infrastructure during future global health emergencies. Methods This paper outlines the features and functions of previously existing and newly created evidence sources routinely searched as part of the NCCMT’s Rapid Evidence Service methods, including databases, platforms, and repositories. Specific functions of each evidence source were assessed as they pertain to searching in the context of a public health emergency, including the topics of indexed citations, the level of evidence of indexed citations, and specific usability features of each evidence source. Results Thirteen evidence sources were assessed, of which four were newly created and nine were either pre-existing or adapted from previously existing resources. Evidence sources varied in topics indexed, level of evidence indexed, and specific searching functions. Conclusion This paper offers insights into which features enabled systematic searching for the completion of rapid reviews to inform decision makers within 5–10 days. These findings provide guidance for knowledge management strategies and evidence infrastructures during future public health emergencies.

Background: As evidence related to the COVID-19 pandemic surged, databases, platforms, and repositories evolved with features and functions to assist users in promptly finding the most relevant evidence. In response, research synthesis teams adopted novel searching strategies to sift through the vast amount of evidence to synthesize and disseminate the most up-to-date evidence. This paper explores the key database features that facilitated systematic searching for rapid evidence synthesis during the COVID-19 pandemic to inform knowledge management infrastructure during future global health emergencies. Methods: This paper outlines the features and functions of previously existing and newly created evidence sources routinely searched as part of the NCCMT's Rapid Evidence Service methods, including databases, platforms, and repositories. Specific functions of each evidence source were assessed as they pertain to searching in the context of a public health emergency, including the topics of indexed citations, the level of evidence of indexed citations, and specific usability features of each evidence source. Results: Thirteen evidence sources were assessed, of which four were newly created and nine were either pre-existing or adapted from previously existing resources. Evidence sources varied in topics indexed, level of evidence indexed, and specific searching functions. Conclusion: This paper offers insights into which features enabled systematic searching for the completion of rapid reviews to inform decision makers within 5–10 days. These findings provide guidance for knowledge management strategies and evidence infrastructures during future public health emergencies. [ABSTRACT FROM AUTHOR]

Abstract Background Demand for rapid evidence-based syntheses to inform health policy and systems decision-making has increased worldwide, including in low- and middle-income countries (LMICs). To promote use of rapid syntheses in LMICs, the WHO’s Alliance for Health Policy and Systems Research (AHPSR) created the Embedding Rapid Reviews in Health Systems Decision-Making (ERA) Initiative. Following a call for proposals, four LMICs were selected (Georgia, India, Malaysia and Zimbabwe) and supported for 1 year to embed rapid response platforms within a public institution with a health policy or systems decision-making mandate. Methods While the selected platforms had experience in health policy and systems research and evidence syntheses, platforms were less confident conducting rapid evidence syntheses. A technical assistance centre (TAC) was created from the outset to develop and lead a capacity-strengthening program for rapid syntheses, tailored to the platforms based on their original proposals and needs as assessed in a baseline questionnaire. The program included training in rapid synthesis methods, as well as generating synthesis demand, engaging knowledge users and ensuring knowledge uptake. Modalities included live training webinars, in-country workshops and support through phone, email and an online platform. LMICs provided regular updates on policy-makers’ requests and the rapid products provided, as well as barriers, facilitators and impacts. Post-initiative, platforms were surveyed. Results Platforms provided rapid syntheses across a range of AHPSR themes, and successfully engaged national- and state-level policy-makers. Examples of substantial policy impact were observed, including for COVID-19. Although the post-initiative survey response rate was low, three quarters of those responding felt confident in their ability to conduct a rapid evidence synthesis. Lessons learned coalesced around three themes – the importance of context-specific expertise in conducting reviews, facilitating cross-platform learning, and planning for platform sustainability. Conclusions The ERA initiative successfully established rapid response platforms in four LMICs. The short timeframe limited the number of rapid products produced, but there were examples of substantial impact and growing demand. We emphasize that LMICs can and should be involved not only in identifying and articulating needs but as co-designers in their own capacity-strengthening programs. More time is required to assess whether these platforms will be sustained for the long-term.

The Cochrane Rapid Review Methods Group (RRMG) first released interim guidance in March 2020 to support authors in conducting rapid reviews (RRs). The objective of this mixed‐methods study was to assess the adherence and investigate authors' understanding of the RRMG guidance. We identified all documents citing the Interim Cochrane RRMG guidance up to February 17, 2022 and performed an exploratory adherence analysis. We interviewed 20 RR authors to assess the recommendations' comprehensibility and reasons for any deviations. Further, we surveyed nine authors of COVID‐19‐related Cochrane reviews for their reasons for not conducting a RR. We analyzed 128 RRs (111 non‐Cochrane, 17 Cochrane) that cited the RRMG guidance. Several recommendations were not followed by a large proportion of RR authors such as stepwise approach to study design inclusion or peer review of search strategies, whereas others were exceeded, for example, dual independent screening of abstracts/full texts. The most reported reasons for deviating from the guidance were time constraints, unclarities in the recommended approach, or inapplicability to the specific RR. Overall, the guidance was viewed as user‐friendly; however, without pre‐existing knowledge of systematic review (SR) conduct, the application was perceived as difficult. The main reasons for conducting a full SR over a RR were late availability of the guidance, preset mandate to conduct a SR, uncertainty regarding methodological distinctions between SR and RR, and inapplicability to the evidence base. Clarifications are warranted throughout the Interim Cochrane RRMG guidance to ensure that users with various experience levels can understand and apply its recommendations accordingly. [ABSTRACT FROM AUTHOR]

Background: Demand for rapid evidence-based syntheses to inform health policy and systems decision-making has increased worldwide, including in low- and middle-income countries (LMICs). To promote use of rapid syntheses in LMICs, the WHO's Alliance for Health Policy and Systems Research (AHPSR) created the Embedding Rapid Reviews in Health Systems Decision-Making (ERA) Initiative. Following a call for proposals, four LMICs were selected (Georgia, India, Malaysia and Zimbabwe) and supported for 1 year to embed rapid response platforms within a public institution with a health policy or systems decision-making mandate. Methods: While the selected platforms had experience in health policy and systems research and evidence syntheses, platforms were less confident conducting rapid evidence syntheses. A technical assistance centre (TAC) was created from the outset to develop and lead a capacity-strengthening program for rapid syntheses, tailored to the platforms based on their original proposals and needs as assessed in a baseline questionnaire. The program included training in rapid synthesis methods, as well as generating synthesis demand, engaging knowledge users and ensuring knowledge uptake. Modalities included live training webinars, in-country workshops and support through phone, email and an online platform. LMICs provided regular updates on policy-makers' requests and the rapid products provided, as well as barriers, facilitators and impacts. Post-initiative, platforms were surveyed. Results: Platforms provided rapid syntheses across a range of AHPSR themes, and successfully engaged national- and state-level policy-makers. Examples of substantial policy impact were observed, including for COVID-19. Although the post-initiative survey response rate was low, three quarters of those responding felt confident in their ability to conduct a rapid evidence synthesis. Lessons learned coalesced around three themes – the importance of context-specific expertise in conducting reviews, facilitating cross-platform learning, and planning for platform sustainability. Conclusions: The ERA initiative successfully established rapid response platforms in four LMICs. The short timeframe limited the number of rapid products produced, but there were examples of substantial impact and growing demand. We emphasize that LMICs can and should be involved not only in identifying and articulating needs but as co-designers in their own capacity-strengthening programs. More time is required to assess whether these platforms will be sustained for the long-term. [ABSTRACT FROM AUTHOR]

Abstract Background Due to the explosion in rapid reviews in the literature during COVID-19, their utility in universal health coverage and in other routine situations, there is now a need to document and further advance the application of rapid review methods, particularly in low-resource settings where a scarcity of resources may preclude the production of a full systematic review. This is the introductory article for a series of articles to further the discussion of rapid reviews for health policy and systems decision-making. Main body The series of papers builds on a practical guide on the conduct and reporting of rapid reviews that was published in 2019. The first paper provides an evaluation of a rapid review platform that was implemented in four centers in low-resource settings, the second paper presents approaches to tailor the methods for decision-makers through rapid reviews, the third paper focuses on selecting different types of rapid review products, and the fourth pertains to reporting the results from a rapid review. Conclusion Rapid reviews have a great potential to inform universal health coverage and global health security interventions, moving forward, including preparedness and response plans to future pandemics. This series of articles will be useful for both researchers leading rapid reviews, as well as decision-makers using the results from rapid reviews.

Various guidelines are available to conduct rapid reviews. However, inconsistencies regarding the methodological practices are prevalent. Following a literature review process, the following databases were searched for guidelines on conducting rapid reviews in psychology: Academic Search Complete, CINAHL, Cochrane Database of Systematic Reviews, Directory of Open Access Journals, Health Source: Nursing/Academic Edition, MEDLINE, PsycArticles, PsycInfo, and SocINDEX. Seventeen articles were included and thematically analysed into eight steps: (i) Stating the aim of the rapid review and formulating the research question; (ii) Setting the eligibility criteria and inclusion and exclusion criteria; (iii) Formulating the search strategy; (iv) Screening and selection of literature; (v) Extraction of data from included literature; (vi) Quality and risk of bias assessment of included literature; (vii) Synthesis and analysis of the included literature; and (viii) Stating the limitations of the rapid review. The eight steps serve as the proposed guideline to conduct rapid reviews in psychology research. [ABSTRACT FROM AUTHOR]

Background: The COVID-19 pandemic highlighted gaps in the public health workforce's capacity to deploy digital technologies while upholding ethical, social justice, and health equity principles. Existing public health competency frameworks have not been updated to reflect the prominent role digital technologies play in contemporary public health, and public health training institutions are seeking to integrate digital technologies in their curricula., Objective: As a first step in a multiphase study exploring recommendations for updates to public health competency frameworks within the Canadian public health context, we conducted a rapid review of literature aiming to identify recommendations for digital competencies, training approaches, and inter- or transdisciplinary partnerships that can enhance public health practitioners' capacity to support the digital transformation of public health., Methods: Following the World Health Organization's (2017) guidelines for rapid reviews, a systematic search was conducted on Ovid MEDLINE, Ovid Embase, ERIC (Education Resources Information Center), and Web of Science for peer-reviewed articles. We also searched Google Scholar and various public health agency and public health association websites for gray literature using search terms related to public health, digital health, practice competencies, and training approaches. We included articles with explicit practice competencies and training recommendations related to digital technologies among public health practitioners published between January 2010 and December 2022. We excluded articles describing these concepts in passing or from a solely clinical perspective., Results: Our search returned 2023 titles and abstracts, of which only 12 studies met the inclusion criteria. We found recommendations for new competencies to enable public health practitioners to appropriately use digital technologies that cut across all existing categories of the core competencies for public health framework of the Public Health Agency of Canada. We also identified a new competency category related to data, data systems management, and governance. Training approaches identified include adapted degree-awarding programs like combined public health and informatics or data science degree programs and ongoing professional certifications with integration of practice-based learning in multi- and interdisciplinary training. Disciplines suggested as important to facilitate practice competency and training recommendations included public health, public health informatics, data, information and computer sciences, biostatistics, health communication, and business., Conclusions: Despite the growth of digital technologies in public health, recommendations about practice competencies and training approaches necessary to effectively support the digital transformation of public health remain limited in the literature. Where available, evidence suggests the workforce requires new competencies that cut across and extend existing public health competencies, including new competencies related to the use and protection of new digital data sources, alongside facilitating health communication and promotion functions using digital media. Recommendations also emphasize the need for training approaches that focus on interdisciplinarity through adapted degree-awarding public health training programs and ongoing professional development., (© Swathi Ramachandran, Hsiu-Ju Chang, Catherine Worthington, Andre Kushniruk, Francisco Ibáñez-Carrasco, Hugh Davies, Geoffrey McKee, Adalsteinn Brown, Mark Gilbert, Ihoghosa Iyamu. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org).)

Background and Objective: Rapid reviews have gained popularity as a pragmatic approach to synthesize evidence in a timely manner to inform decision-making in healthcare. This article provides an overview of the key concepts and methodological considerations in conducting rapid reviews, drawing from a series of recently published guidance papers by the Cochrane Rapid Reviews Methods Group., Study Design and Setting: We discuss the definition, characteristics, and potential applications of rapid reviews and the trade-offs between speed and rigor. We present a practical example of a rapid review and highlight the methodological considerations outlined in the updated Cochrane guidance, including recommendations for literature searching, study selection, data extraction, risk of bias assessment, synthesis, and assessing the certainty of evidence., Results: Rapid reviews can be a valuable tool for evidence-based decision-making, but it is essential to understand their limitations and adhere to methodological standards to ensure their validity and reliability., Conclusion: As the demand for rapid evidence synthesis continues to grow, further research is needed to refine and standardize the methods and reporting of rapid reviews., Plain Language Summary: Rapid reviews are a type of research method designed to quickly gather and summarize evidence to support decision-making in healthcare. They are particularly useful when timely information is needed, such as during a public health emergency. This article explains the key aspects of how rapid reviews are conducted, based on the latest guidance from experts. Rapid reviews involve several steps, including searching for relevant studies, selecting which studies to include, and carefully examining the quality of the evidence. Although rapid reviews are faster to complete than full systematic reviews, they still follow rigorous processes to ensure that the findings are reliable. This article also provides an example of a rapid review in action, demonstrating how these reviews can be applied in real-world situations. While rapid reviews are a powerful tool for making quick, evidence-based decisions, it is important to be aware of their limitations. Researchers must follow established methods to make sure the results are as accurate and useful as possible. As more people use rapid reviews, ongoing research is needed to improve and standardize how they are done., Competing Interests: Declaration of competing interest D. D., C. H., G. G., B. N. S., and U. G. are co-convenors of the Cochrane Rapid Reviews Methods Group., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Background: Due to the explosion in rapid reviews in the literature during COVID-19, their utility in universal health coverage and in other routine situations, there is now a need to document and further advance the application of rapid review methods, particularly in low-resource settings where a scarcity of resources may preclude the production of a full systematic review. This is the introductory article for a series of articles to further the discussion of rapid reviews for health policy and systems decision-making. Main body: The series of papers builds on a practical guide on the conduct and reporting of rapid reviews that was published in 2019. The first paper provides an evaluation of a rapid review platform that was implemented in four centers in low-resource settings, the second paper presents approaches to tailor the methods for decision-makers through rapid reviews, the third paper focuses on selecting different types of rapid review products, and the fourth pertains to reporting the results from a rapid review. Conclusion: Rapid reviews have a great potential to inform universal health coverage and global health security interventions, moving forward, including preparedness and response plans to future pandemics. This series of articles will be useful for both researchers leading rapid reviews, as well as decision-makers using the results from rapid reviews. [ABSTRACT FROM AUTHOR]

Abstract Background Accepted systematic review (SR) methodology requires citation screening by two reviewers to maximise retrieval of eligible studies. We hypothesized that records could be excluded by a single reviewer without loss of sensitivity in two conditions; the record was ineligible for multiple reasons, or the record was ineligible for one or more specific reasons that could be reliably assessed. Methods Twenty-four SRs performed at CHEO, a pediatric health care and research centre in Ottawa, Canada, were divided into derivation and validation sets. Exclusion criteria during abstract screening were sorted into 11 specific categories, with loss in sensitivity determined by individual category and by number of exclusion criteria endorsed. Five single reviewer algorithms that combined individual categories and multiple exclusion criteria were then tested on the derivation and validation sets, with success defined a priori as less than 5% loss of sensitivity. Results The 24 SRs included 930 eligible and 27390 ineligible citations. The reviews were mostly focused on pediatrics (70.8%, N=17/24), but covered various specialties. Using a single reviewer to exclude any citation led to an average loss of sensitivity of 8.6% (95%CI, 6.0–12.1%). Excluding citations with ≥2 exclusion criteria led to 1.2% average loss of sensitivity (95%CI, 0.5–3.1%). Five specific exclusion criteria performed with perfect sensitivity: conference abstract, ineligible age group, case report/series, not human research, and review article. In the derivation set, the five algorithms achieved a loss of sensitivity ranging from 0.0 to 1.9% and work-saved ranging from 14.8 to 39.1%. In the validation set, the loss of sensitivity for all 5 algorithms remained below 2.6%, with work-saved between 10.5% and 48.2%. Conclusions Findings suggest that targeted application of single-reviewer screening, considering both type and number of exclusion criteria, could retain sensitivity and significantly decrease workload. Further research is required to investigate the potential for combining this approach with crowdsourcing or machine learning methodologies.

Abstract Background The COVID-19 pandemic has created a sense of urgency in the research community in their bid to contribute to the evidence required for healthcare policy decisions. With such urgency, researchers experience methodological challenges to maintain the rigour and transparency of their work. With this in mind, we offer reflections on our recent experience of undertaking a rapid Cochrane qualitative evidence synthesis (QES). Methods This process paper, using a reflexive approach, describes a rapid QES prepared during, and in response to, the COVID-19 pandemic. Findings This paper reports the methodological decisions we made and the process we undertook. We place our decisions in the context of guidance offered in relation to rapid reviews and previously conducted QESs. We highlight some of the challenges we encountered in finding the balance between the time needed for thoughtfulness and comprehensiveness whilst providing a rapid response to an urgent request for evidence. Conclusion The need for more guidance on rapid QES remains, but such guidance needs to be based on actual worked examples and case studies. This paper and the reflections offered may provide a useful framework for others to use and further develop.

Abstract Background Systematic reviews often require substantial resources, partially due to the large number of records identified during searching. Although artificial intelligence may not be ready to fully replace human reviewers, it may accelerate and reduce the screening burden. Using DistillerSR (May 2020 release), we evaluated the performance of the prioritization simulation tool to determine the reduction in screening burden and time savings. Methods Using a true recall @ 95%, response sets from 10 completed systematic reviews were used to evaluate: (i) the reduction of screening burden; (ii) the accuracy of the prioritization algorithm; and (iii) the hours saved when a modified screening approach was implemented. To account for variation in the simulations, and to introduce randomness (through shuffling the references), 10 simulations were run for each review. Means, standard deviations, medians and interquartile ranges (IQR) are presented. Results Among the 10 systematic reviews, using true recall @ 95% there was a median reduction in screening burden of 47.1% (IQR: 37.5 to 58.0%). A median of 41.2% (IQR: 33.4 to 46.9%) of the excluded records needed to be screened to achieve true recall @ 95%. The median title/abstract screening hours saved using a modified screening approach at a true recall @ 95% was 29.8 h (IQR: 28.1 to 74.7 h). This was increased to a median of 36 h (IQR: 32.2 to 79.7 h) when considering the time saved not retrieving and screening full texts of the remaining 5% of records not yet identified as included at title/abstract. Among the 100 simulations (10 simulations per review), none of these 5% of records were a final included study in the systematic review. The reduction in screening burden to achieve true recall @ 95% compared to @ 100% resulted in a reduced screening burden median of 40.6% (IQR: 38.3 to 54.2%). Conclusions The prioritization tool in DistillerSR can reduce screening burden. A modified or stop screening approach once a true recall @ 95% is achieved appears to be a valid method for rapid reviews, and perhaps systematic reviews. This needs to be further evaluated in prospective reviews using the estimated recall.

Abstract Background Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice. Methods We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere. Results Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n = 99) and described methods to conduct the RR (91%; n = 94), while several addressed political or health systems contexts (61%; n = 63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n = 68), yet only 32% (n = 33) involved their tacit knowledge, while fewer (27%; n = 28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n = 24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n = 26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read. Conclusions Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored. Protocol availability The protocol, published on the Open Science Framework, is available at: osf.io/68tj7

ABSTRACT BACKGROUND: COVID-19 infection has high transmissibility and several measures have been adopted for controlling its dissemination. OBJECTIVE: To identify and summarize the evidence from Cochrane systematic reviews (SRs) regarding measures for controlling the dissemination of COVID-19 infection. DESIGN AND SETTING: This review of Cochrane SRs was carried out in the Division of Vascular and Endovascular Surgery and in the Division of Emergency Medicine and Evidence-Based Medicine of Universidade Federal de São Paulo, Brazil. METHODS: A comprehensive search in the Cochrane Database of Systematic Reviews retrieved all Cochrane SRs directly related to measures for controlling COVID-19 dissemination. The main characteristics and results of all the SRs included were summarized and discussed. RESULTS: Three Cochrane SRs were included in the qualitative synthesis. These evaluated population-based and individual measures for controlling the dissemination of COVID-19. CONCLUSION: Low-certainty evidence shows that quarantine for people exposed to confirmed or suspected COVID-19 cases prevented 44% to 81% of incident cases and 31% to 63% of deaths, compared with situations of no measures. Moreover, the sooner the quarantine measures were implemented, the greater the cost savings were. High-confidence evidence showed that clear communication about infection control and prevention guidelines was vital for successful implementation. Low-certainty evidence showed that healthcare professionals with long gowns were less exposed to contamination than were those using coveralls. In addition, coveralls were more difficult to doff. Further SRs on controlling the dissemination of COVID-19 infection are desirable.

Abstract Background We investigated the feasibility of using a machine learning tool’s relevance predictions to expedite title and abstract screening. Methods We subjected 11 systematic reviews and six rapid reviews to four retrospective screening simulations (automated and semi-automated approaches to single-reviewer and dual independent screening) in Abstrackr, a freely-available machine learning software. We calculated the proportion missed, workload savings, and time savings compared to single-reviewer and dual independent screening by human reviewers. We performed cited reference searches to determine if missed studies would be identified via reference list scanning. Results For systematic reviews, the semi-automated, dual independent screening approach provided the best balance of time savings (median (range) 20 (3–82) hours) and reliability (median (range) proportion missed records, 1 (0–14)%). The cited references search identified 59% (n = 10/17) of the records missed. For the rapid reviews, the fully and semi-automated approaches saved time (median (range) 9 (2–18) hours and 3 (1–10) hours, respectively), but less so than for the systematic reviews. The median (range) proportion missed records for both approaches was 6 (0–22)%. Conclusion Using Abstrackr to assist one of two reviewers in systematic reviews saves time with little risk of missing relevant records. Many missed records would be identified via other means.

Abstract Background Rapid reviews (RRs) have emerged as an efficient alternative to time-consuming systematic reviews—they can help meet the demand for accelerated evidence synthesis to inform decision-making in healthcare. The synthesis of diagnostic evidence has important methodological challenges. Here, we performed an international survey to identify the current practice of producing RRs for diagnostic tests. Methods We developed and administered an online survey inviting institutions that perform RRs of diagnostic tests from all over the world. Results All participants (N = 25) reported the implementation of one or more methods to define the scope of the RR; however, only one strategy (defining a structured question) was used by ≥90% of participants. All participants used at least one methodological shortcut including the use of a previous review as a starting point (92%) and the use of limits on the search (96%). Parallelization and automation of review tasks were not extensively used (48 and 20%, respectively). Conclusion Our survey indicates a greater use of shortcuts and limits for conducting diagnostic test RRs versus the results of a recent scoping review analyzing published RRs. Several shortcuts are used without knowing how their implementation affects the results of the evidence synthesis in the setting of diagnostic test reviews. Thus, a structured evaluation of the challenges and implications of the adoption of these RR methods is warranted.

The Cochrane COVID‐19 Study Register (CCSR) is a public, continually updated database of COVID‐19 study references. The aim of this study‐based register is to support rapid and living evidence synthesis, including an evidence ecosystem of COVID‐19 research (CEOsys). In November and December 2020, we conducted an evaluation of the CCSR for CEOsys, measured its performance and identified areas for improvement. For the evaluation we generated a purposive sample of 286 studies from 20 reviews to calculate the CCSR's comprehensiveness (sensitivity), accuracy (correctly classified and linked studies) and currency (time to publish and process references). Our sample showed that the CCSR had an overall comprehensiveness of 77.2%, with the highest coverage for interventional studies (94.4%). The study register had 100% coverage for trial registry records, 86.5% for journal articles and 52.4% for preprints. A total of 98.3% of references were correctly classified with regard to study type, and 93.4% with regard to study aim. A total of 89% of studies were correctly linked. A total of 81.4% of references were published to the register in under 30 days, with 0.5 day (median) for trial registry records, 2 days for journal articles and 56 days for preprints. The CCSR had high comprehensiveness, accurate study classifications and short publishing times for journal articles and trial registry records in the sample. We identified that coverage and publishing time for preprints needed improvement. Finally, the evaluation illustrated the value of a study‐based register for identifying additional study references for analysis in evidence synthesis. [ABSTRACT FROM AUTHOR]

During a pandemic, clinicians and scientists should be able to identify the emerging scientific evidence as efficiently as possible. A study-based register is essential for this. It should be up to date, evaluate important primary databases and increase the findability of studies by classifying them according to study characteristics. In early 2020, the surge in publications on the SARS-CoV-2/COVID-19 pandemic created an urgent need for a structured database to support the rapid development of evidence-based recommendations. This short article introduces the Cochrane COVID-19 trial registry, which was developed by Cochrane in April 2020 and has been continuously refined over the past 14 months.

Although systematic reviews are the method of choice to synthesize scientific evidence, they can take years to complete and publish. Clinicians, managers, and policy-makers often need input from scientific evidence in a more timely and resource-efficient manner. For this purpose, rapid reviews are conducted. Rapid reviews are performed using an accelerated process. However, they should not be less systematic than standard systematic reviews, and the introduction of bias must be avoided. In this article, we describe what rapid reviews are, present their characteristics, give some examples, highlight potential pitfalls, and draw attention to the importance of evidence summaries in order to facilitate adoption in clinical decision-making. [ABSTRACT FROM AUTHOR]

Abstract Background Web applications that employ natural language processing technologies to support systematic reviewers during abstract screening have become more common. The goal of our project was to conduct a case study to explore a screening approach that temporarily replaces a human screener with a semi-automated screening tool. Methods We evaluated the accuracy of the approach using DistillerAI as a semi-automated screening tool. A published comparative effectiveness review served as the reference standard. Five teams of professional systematic reviewers screened the same 2472 abstracts in parallel. Each team trained DistillerAI with 300 randomly selected abstracts that the team screened dually. For all remaining abstracts, DistillerAI replaced one human screener and provided predictions about the relevance of records. A single reviewer also screened all remaining abstracts. A second human screener resolved conflicts between the single reviewer and DistillerAI. We compared the decisions of the machine-assisted approach, single-reviewer screening, and screening with DistillerAI alone against the reference standard. Results The combined sensitivity of the machine-assisted screening approach across the five screening teams was 78% (95% confidence interval [CI], 66 to 90%), and the combined specificity was 95% (95% CI, 92 to 97%). By comparison, the sensitivity of single-reviewer screening was similar (78%; 95% CI, 66 to 89%); however, the sensitivity of DistillerAI alone was substantially worse (14%; 95% CI, 0 to 31%) than that of the machine-assisted screening approach. Specificities for single-reviewer screening and DistillerAI were 94% (95% CI, 91 to 97%) and 98% (95% CI, 97 to 100%), respectively. Machine-assisted screening and single-reviewer screening had similar areas under the curve (0.87 and 0.86, respectively); by contrast, the area under the curve for DistillerAI alone was just slightly better than chance (0.56). The interrater agreement between human screeners and DistillerAI with a prevalence-adjusted kappa was 0.85 (95% CI, 0.84 to 0.86%). Conclusions The accuracy of DistillerAI is not yet adequate to replace a human screener temporarily during abstract screening for systematic reviews. Rapid reviews, which do not require detecting the totality of the relevant evidence, may find semi-automation tools to have greater utility than traditional systematic reviews.

Abstract Background Assessment of diagnostic tests, broadly defined as any element that aids in the collection of additional information for further clarification of a patient’s health status, has increasingly become a critical issue in health policy and decision-making. Diagnostic evidence, including the accuracy of a medical test for a target condition, is commonly appraised using standard systematic review methodology. Owing to the considerable time and resources required to conduct these, rapid reviews have emerged as a pragmatic alternative by tailoring methods according to the decision maker’s circumstances. However, it is not known if streamlining methodological aspects has an impact on the validity of evidence synthesis. Furthermore, due to the particular nature and complexity of the appraisal of diagnostic accuracy, there is need for detailed guidance on how to conduct rapid reviews of diagnostic tests. In this study, we aim to identify the methods currently used by rapid review developers to synthesize evidence on diagnostic test accuracy, as well as to analyze potential shortcomings and challenges related to these methods. Methods We will carry out a two-fold approach: (1) an international survey of professionals working in organizations that develop rapid reviews of diagnostic tests, in terms of the methods and resources used by these agencies when conducting rapid reviews, and (2) semi-structured interviews with senior-level individuals to further explore and validate the findings from the survey and to identify challenges in conducting rapid reviews. We will use STATA 15.0 for quantitative analyses and framework analysis for qualitative analyses. We will ensure protection of data during all stages. Discussion The main result of this research will be a map of methods and resources currently used for conducting rapid reviews of diagnostic test accuracy, as well as methodological shortcomings and potential solutions in diagnostic knowledge synthesis that require further research.

The short time frame associated with rapid reviews can be challenging for researchers conducting qualitative evidence synthesis. In these circumstances a Best‐Fit Framework Synthesis, drawing upon existing theory and/or research, may be conducted to rapidly make sense of qualitative evidence. This article discusses a "Rapid Best‐Fit" approach to conducting Framework Synthesis within an 6‐week rapid systematic review of qualitative evidence. In the absence of a suitable theoretical model to inform the synthesis, we drew upon our research objectives to structure a framework, and to identify the studies which contained the most relevant data. Themes from these studies were used to revise the initial framework before inductive thematic synthesis finalized theme content. This "Rapid Best‐Fit" approach yielded results aligned with the needs of the commissioners of the review and is a useful addition to qualitative synthesis methods, allowing for the incorporation of diverse views and experiences into fast‐paced decision making scenarios. [ABSTRACT FROM AUTHOR]

Background: Accepted systematic review (SR) methodology requires citation screening by two reviewers to maximise retrieval of eligible studies. We hypothesized that records could be excluded by a single reviewer without loss of sensitivity in two conditions; the record was ineligible for multiple reasons, or the record was ineligible for one or more specific reasons that could be reliably assessed. Methods: Twenty-four SRs performed at CHEO, a pediatric health care and research centre in Ottawa, Canada, were divided into derivation and validation sets. Exclusion criteria during abstract screening were sorted into 11 specific categories, with loss in sensitivity determined by individual category and by number of exclusion criteria endorsed. Five single reviewer algorithms that combined individual categories and multiple exclusion criteria were then tested on the derivation and validation sets, with success defined a priori as less than 5% loss of sensitivity. Results: The 24 SRs included 930 eligible and 27390 ineligible citations. The reviews were mostly focused on pediatrics (70.8%, N=17/24), but covered various specialties. Using a single reviewer to exclude any citation led to an average loss of sensitivity of 8.6% (95%CI, 6.0–12.1%). Excluding citations with ≥2 exclusion criteria led to 1.2% average loss of sensitivity (95%CI, 0.5–3.1%). Five specific exclusion criteria performed with perfect sensitivity: conference abstract, ineligible age group, case report/series, not human research, and review article. In the derivation set, the five algorithms achieved a loss of sensitivity ranging from 0.0 to 1.9% and work-saved ranging from 14.8 to 39.1%. In the validation set, the loss of sensitivity for all 5 algorithms remained below 2.6%, with work-saved between 10.5% and 48.2%. Conclusions: Findings suggest that targeted application of single-reviewer screening, considering both type and number of exclusion criteria, could retain sensitivity and significantly decrease workload. Further research is required to investigate the potential for combining this approach with crowdsourcing or machine learning methodologies. [ABSTRACT FROM AUTHOR]

Despite the surge in popularity of virtual health care services as a means of delivering health care through technology, the integration of research evidence into practice remains a challenge. Rapid reviews, a type of time-efficient evidence synthesis, offer a potential solution to bridge the gap between knowledge and action. This paper aims to highlight the experiences of the Fraser Health Authority's Virtual Health team in conducting rapid reviews. This paper discusses the experiences of the Virtual Health team in conducting 15 rapid reviews over the course of 1.5 years and the benefit of involving diverse stakeholders including researchers, project and clinical leads, and students for the creation of user-friendly knowledge products to summarize results. The Virtual Health team found rapid reviews to be a valuable tool for evidence-informed decision-making in virtual health care. Involving stakeholders and focusing on implementation considerations are crucial for maximizing the impact of rapid reviews. Health care decision makers are encouraged to consider implementing rapid review processes to improve the translation of research evidence into practice, ultimately enhancing patient outcomes and promoting a culture of evidence-informed care., (©Megan MacPherson, Sarah Rourke. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 22.05.2024.)

Abstract Background Guidelines in the healthcare field generally should contain evidence-based recommendations to inform healthcare decisions. Guidelines often require 2 years or more to develop, but certain circumstances necessitate the development of rapid guidelines (RGs) in a short period of time. Upholding methodological rigor while meeting the reduced development timeframe presents a challenge for developing RGs. Our objective was to review current practices and standards for the development of RGs. This is the first of a series of three articles addressing methodological issues around RGs. Methods We conducted a systematic survey of methods manuals and published RGs to identify reasons for the development of RGs. Data sources included existing guideline manuals, published RGs, Trip Medical Database, MEDLINE, EMBASE and communication with guideline developers until February 2018. Results We identified 46 guidelines that used a shortened timeframe for their development. Nomenclature describing RGs varied across organisations, wherein the United States Centers for Disease Control and Prevention produced ‘Interim Guidelines’, the National Institute for Health and Care Excellence in the United Kingdom developed ‘Short Clinical Guidelines’, and WHO provided ‘Rapid Advice’. The rationale for RGs included response to emergencies, rapid increases in cases of a condition or disease severity, or new evidence regarding treatment. In general, the methods to assess the quality of evidence, the consensus process and the management of the conflict of interest were not always clear. While we identified another 11 RGs from other institutions, there was no reference to timeframe and reasons for conducting a RG. The three organisations mentioned above provide guidance for the development of RGs. Conclusions There is a lack of standardised nomenclature and definitions regarding RGs and there is inconsistency in the methods described in manuals and in RG. It is therefore important that all RGs provide a detailed and transparent description of their methods in order for readers and end-users to be able to assess their quality and validate their findings.

Abstract Background Situations such as public health emergencies and outbreaks necessitate the development and publication of high-quality recommendations within a condensed timeframe. For example, WHO has produced examples of and guidance for the development of rapid guidelines (RGs). However, more information is needed to understand the experiences and perceptions of guideline developers. This is the second of a series of three articles addressing methodological issues around RGs. This study describes the perceptions and experiences of guideline developers at WHO about RGs. Methods We conducted interviews consisting of open- and closed-ended questions with guideline developers at WHO. Our analysis described the definition and rationale of RGs, the differences from regular guidelines with regard to timelines from topic definition until publication, barriers to identifying the evidence and the lack of a standard methodology to develop RGs. Results We interviewed 10 participants, the majority of whom were comfortable with the current WHO definition of RGs. Most stated that the rationale for developing RGs should be in response to new evidence about efficacy, cost-effectiveness or safety. Respondents differed with regards to the amount of time RGs should take. While the majority of participants agreed that guidelines should be based on a systematic review, this step in the process was considered the most time and resource intensive. Challenges for developing RGs included limited personnel and financial resources as well as the lack of evidence. Facilitators, in turn, that may improve RG development include additional financial and personnel resources as well as the use of virtual meetings. Conclusions While our study suggests a strong need and rationale for the development of RGs, standardisation of timelines and guidance on panel composition, peer-review process, conduct of meetings and sources of permissible evidence require further research.

Abstract Background Rapid response in health technology assessment is a synthesis of the best available evidence prepared in a timely manner to meet specific demands. We build a consensus among Brazilian specialists in health technology assessment to propose guidelines for the development of rapid response. Methods Based on a systematic review that proposed eight methodological steps to conduct rapid response, we applied a modified Delphi technique (without open questions in the first round) to reach consensus among Brazilian experts in health technology assessment. Twenty participants were invited to judge the feasibility of each methodological step in a five-point Likert scale. Consensus was reached if the step had 70% positive approval or interquartile range ≤ 1. Results The achievement of consensus was reached in the second round. Between the first and the second round, we scrutinized all points reported by the experts. The Delphi panel reached consensus of eight steps: definition of the structured question of rapid response (with a restricted scope); definition of the eligibility criteria for study types (preferably systematic reviews); search strategy (language and data limits) and sources of information (minimum two); selection of studies (independently by two responders); critical appraisal of the included studies and the risk of bias for the outcomes of interest; data extraction from the included articles; summary of evidence; and preparation of the report. Conclusions The guidelines for rapid response in health technology assessment may help governments to make better decisions in a short period of time (35 days). The adoption of methodological processes should improve both the quality and consistency of health technology assessments of rapid decisions in the Brazilian setting.

Background: We investigated the feasibility of using a machine learning tool's relevance predictions to expedite title and abstract screening.Methods: We subjected 11 systematic reviews and six rapid reviews to four retrospective screening simulations (automated and semi-automated approaches to single-reviewer and dual independent screening) in Abstrackr, a freely-available machine learning software. We calculated the proportion missed, workload savings, and time savings compared to single-reviewer and dual independent screening by human reviewers. We performed cited reference searches to determine if missed studies would be identified via reference list scanning.Results: For systematic reviews, the semi-automated, dual independent screening approach provided the best balance of time savings (median (range) 20 (3-82) hours) and reliability (median (range) proportion missed records, 1 (0-14)%). The cited references search identified 59% (n = 10/17) of the records missed. For the rapid reviews, the fully and semi-automated approaches saved time (median (range) 9 (2-18) hours and 3 (1-10) hours, respectively), but less so than for the systematic reviews. The median (range) proportion missed records for both approaches was 6 (0-22)%.Conclusion: Using Abstrackr to assist one of two reviewers in systematic reviews saves time with little risk of missing relevant records. Many missed records would be identified via other means. [ABSTRACT FROM AUTHOR]

Background: Rapid reviews (RRs) have emerged as an efficient alternative to time-consuming systematic reviews-they can help meet the demand for accelerated evidence synthesis to inform decision-making in healthcare. The synthesis of diagnostic evidence has important methodological challenges. Here, we performed an international survey to identify the current practice of producing RRs for diagnostic tests.Methods: We developed and administered an online survey inviting institutions that perform RRs of diagnostic tests from all over the world.Results: All participants (N = 25) reported the implementation of one or more methods to define the scope of the RR; however, only one strategy (defining a structured question) was used by ≥90% of participants. All participants used at least one methodological shortcut including the use of a previous review as a starting point (92%) and the use of limits on the search (96%). Parallelization and automation of review tasks were not extensively used (48 and 20%, respectively).Conclusion: Our survey indicates a greater use of shortcuts and limits for conducting diagnostic test RRs versus the results of a recent scoping review analyzing published RRs. Several shortcuts are used without knowing how their implementation affects the results of the evidence synthesis in the setting of diagnostic test reviews. Thus, a structured evaluation of the challenges and implications of the adoption of these RR methods is warranted. [ABSTRACT FROM AUTHOR]

Abstract Background Systematic reviews offer the most reliable and valid support for health policy decision-making, patient information, and guideline development. However, they are labor intensive and frequently take longer than 1 year to complete. Consequently, they often do not meet the needs of those who need to make decisions quickly. Rapid reviews have therefore become a pragmatic alternative to systematic reviews. They are knowledge syntheses that abbreviate certain methodological aspects of systematic reviews to produce information more quickly. Methodological shortcuts often take place in literature identification. A potential drawback is less reliable results. To date, the impact of abbreviated searches on estimates of treatment effects and subsequent conclusions has not been analyzed systematically across multiple bodies of evidence. We aim to answer the research question: Do bodies of evidence that are based on abbreviated literature searches lead to different conclusions about benefits and harms of interventions compared with bodies of evidence that are based on comprehensive, systematic literature searches? Methods We will use a non-inferiority and meta-epidemiologic design. The primary outcome is the proportion of discordant conclusions based on different search approaches. Drawing of a pool of Cochrane reports published between 2012 and 2016, we will randomly select 60 reports. Eligible reports are those that present a summary-of-findings table, draw a clear conclusion, present data for meta-analyses, and document the search strategy clearly. We will conduct several abbreviated searches to detect whether included studies in these Cochrane reviews could be detected. If searches could not detect all studies, we will revise the original summary-of-findings table and ask review authors whether the missed evidence would change conclusions of their report. We will determine the proportion of discordant conclusions for each abbreviated search approach. We will consider an abbreviated search as non-inferior if the lower limit of the 95% confidence interval of the proportion of discordant conclusions is below the non-inferiority margin, which is determined based on results of a survey for clinical and public health scenarios. Discussion This will be the first study to assess whether the reduced sensitivity of abbreviated searches has an impact on conclusions across multiple bodies of evidence, not only on effect estimates.

Abstract Background Policymakers and healthcare stakeholders are increasingly seeking evidence to inform the policymaking process, and often use existing or commissioned systematic reviews to inform decisions. However, the methodologies that make systematic reviews authoritative take time, typically 1 to 2 years to complete. Outside the traditional SR timeline, “rapid reviews” have emerged as an efficient tool to get evidence to decision-makers more quickly. However, the use of rapid reviews does present challenges. To date, there has been limited published empirical information about this approach to compiling evidence. Thus, it remains a poorly understood and ill-defined set of diverse methodologies with various labels. In recent years, the need to further explore rapid review methods, characteristics, and their use has been recognized by a growing network of healthcare researchers, policymakers, and organizations, several with ties to Cochrane, which is recognized as representing an international gold standard for high-quality, systematic reviews. Purpose In this commentary, we introduce the newly established Cochrane Rapid Reviews Methods Group developed to play a leading role in guiding the production of rapid reviews given they are increasingly employed as a research synthesis tool to support timely evidence-informed decision-making. We discuss how the group was formed and outline the group’s structure and remit. We also discuss the need to establish a more robust evidence base for rapid reviews in the published literature, and the importance of promoting registration of rapid review protocols in an effort to promote efficiency and transparency in research. Conclusion As with standard systematic reviews, the core principles of evidence-based synthesis should apply to rapid reviews in order to minimize bias to the extent possible. The Cochrane Rapid Reviews Methods Group will serve to establish a network of rapid review stakeholders and provide a forum for discussion and training. By facilitating exchange, the group will strive to conduct research to advance the methods of rapid reviews.

Background: There is increasing demand for rapid reviews and timely evidence synthesis. The goal of this project was to understand end-user perspectives on the utility and limitations of rapid products including evidence inventories, rapid responses, and rapid reviews. Methods: Interviews were conducted with key informants representing: guideline developers (n = 3), health care providers/health system organizations (n = 3), research funders (n = 1), and payers/health insurers (n = 1). We elicited perspectives on important characteristics of systematic reviews, acceptable methods to streamline reviews, and uses of rapid products. We analyzed content of the interview transcripts and identified themes and subthemes. Results: Key informants identified the following as critical features of evidence reviews: (1) originating from a reliable source (i.e., conducted by experienced reviewers from an established research organization), (2) addressing clinically relevant questions, and (3) trusted relationship between the user and producer. Key informants expressed strong preference for the following review methods and characteristics: use of evidence tables, quality rating of studies, assessments of total evidence quality/strength, and use of summary tables for results and conclusions. Most acceptable trade-offs to increase efficiencies were limiting the literature search (e.g., limiting search dates or language) and performing single screening of citations and full texts for relevance. Key informants perceived rapid products (particularly evidence inventories and rapid responses) as useful interim products to inform downstream investigation (e.g., whether to proceed with a full review or guideline, direction for future research). Most key informants indicated that evidence analysis/synthesis and quality/strength of evidence assessments were important for decision-making. They reported that rapid reviews in particular were useful for guideline development on narrow topics, policy decisions when a quick turn-around is needed, decision-making for practicing clinicians in nuanced clinical settings, and decisions about coverage by payers/health insurers. Rapid reviews may be more relevant within specific clinical settings or health systems; whereas, broad/national guidelines often need a traditional systematic review. Conclusions: Key informants interviewed in our study indicated that evidence inventories, rapid responses, and rapid reviews have utility in specific decisions and contexts. They indicated that the credibility of the review producer, relevance of key questions, and close working relationship between the end-user and producer are critical for any rapid product. Our findings are limited by the sample size which may have been too small to reach saturation for the themes described. [ABSTRACT FROM AUTHOR]

Background: Rapid reviews are increasingly being used to help policy makers access research in short time frames. A clear articulation of the review's purpose, questions, scope, methods and reporting format is thought to improve the quality and generalisability of review findings. The aim of the study is to explore the effectiveness of knowledge brokering in improving the perceived clarity of rapid review proposals from the perspective of potential reviewers. To conduct the study, we drew on the Evidence Check program, where policy makers draft a review proposal (a pre knowledge brokering proposal) and have a 1-hour session with a knowledge broker, who re-drafts the proposal based on the discussion (a post knowledge brokering proposal). Methods: We asked 30 reviewers who had previously undertaken Evidence Check reviews to examine the quality of 60 pre and 60 post knowledge brokering proposals. Reviewers were blind to whether the review proposals they received were pre or post knowledge brokering. Using a six-point Likert scale, reviewers scored six questions examining clarity of information about the review's purpose, questions, scope, method and format and reviewers' confidence that they could meet policy makers' needs. Each reviewer was allocated two pre and two post knowledge brokering proposals, randomly ordered, from the 60 reviews, ensuring no reviewer received a pre and post knowledge brokering proposal from the same review. Results: The results showed that knowledge brokering significantly improved the scores for all six questions addressing the perceived clarity of the review proposal and confidence in meeting policy makers' needs; with average changes of 0.68 to 1.23 from pre to post across the six domains. Conclusions: This study found that knowledge brokering increased the perceived clarity of information provided in Evidence Check rapid review proposals and the confidence of reviewers that they could meet policy makers' needs. Further research is needed to identify how the knowledge brokering process achieves these improvements and to test the applicability of the findings in other rapid review programs. [ABSTRACT FROM AUTHOR]