Search

IntroductionIdeally, health conditions causing the greatest global disease burden should attract increased research attention. We conducted a comprehensive global study investigating the number of randomised controlled trials (RCTs) published on different health conditions, and how this compares with the global disease burden that they impose.MethodsWe use machine learning to monitor PubMed daily, and find and analyse RCT reports. We assessed RCTs investigating the leading causes of morbidity and mortality from the Global Burden of Disease study. Using regression models, we compared numbers of actual RCTs in different health conditions to numbers predicted from their global disease burden (Disability-Adjusted Life Years [DALYs]). We investigated whether RCT numbers differed for conditions disproportionately affecting countries with lower socio-economic development.ResultsWe estimate 463,000 articles describing RCTs (95% prediction interval 439,000–485,000) were published from 1990 to July 2020. RCTs recruited a median of 72 participants (interquartile range 32–195). 82% of RCTs were conducted by researchers in the top fifth of countries by socio-economic development. As DALYs increased for a particular health condition by 10%, the number of RCTs in the same year increased by 5% (3.2%–6.9%), but the association was weak (adjusted R2=0.13). Conditions disproportionately affecting countries with lower socio-economic development, including respiratory infections and tuberculosis (7 thousand RCTs below predicted) and enteric infections (10 thousand RCTs below predicted), appear relatively under-researched for their disease burden. Each 10% shift in DALYs towards countries with low and middle socio-economic development was associated with a 4% reduction in RCTs (3.7%–4.9%). These disparities have not changed substantially over time.ConclusionResearch priorities are not well optimized to reduce the global burden of disease. Most RCTs are produced by highly developed countries, and the health needs of these countries have been, on average, favoured.Key questionsWhat is already known?Prior studies have manually investigated the relationship between published research in different health conditions and the global burden of disease that they impose.However, these analyses have been mostly limited to estimates of research funding from national funders, or smaller scale analysis of older publication records.These studies have highlighted disparities in research relative to burden, but they are not sufficient to enable global targeting of research to optimise improvements in disease burden.What are the new findings?We automatically process all of PubMed, allowing us to conduct a continually updated, comprehensive analysis of published reports of RCTs, including the number of participants per RCT and the health conditions studied.We found that considerable disparities exist between the relative volume of evidence on some conditions and the global burden of disease that they impose, as calculated by the Global Burden of Disease study.Further, our analysis suggests that there exists a smaller amount of evidence for conditions that impose a comparatively large burden of disease in lower-income countries.What do the new findings imply?Looking at numbers of RCTs published, and the numbers of participants in these trials, it seems that research priorities are not optimized to reduce the global burden of disease, and that research for conditions affecting higher-income countries has, on average, been favoured.The findings from this study could help research funders to focus research investment in areas where the largest reductions in disease burden could be made.

Objective Randomized controlled trials (RCTs) are the gold standard method for evaluating whether a treatment works in health care but can be difficult to find and make use of. We describe the development and evaluation of a system to automatically find and categorize all new RCT reports. Materials and Methods Trialstreamer continuously monitors PubMed and the World Health Organization International Clinical Trials Registry Platform, looking for new RCTs in humans using a validated classifier. We combine machine learning and rule-based methods to extract information from the RCT abstracts, including free-text descriptions of trial PICO (populations, interventions/comparators, and outcomes) elements and map these snippets to normalized MeSH (Medical Subject Headings) vocabulary terms. We additionally identify sample sizes, predict the risk of bias, and extract text conveying key findings. We store all extracted data in a database, which we make freely available for download, and via a search portal, which allows users to enter structured clinical queries. Results are ranked automatically to prioritize larger and higher-quality studies. Results As of early June 2020, we have indexed 673 191 publications of RCTs, of which 22 363 were published in the first 5 months of 2020 (142 per day). We additionally include 304 111 trial registrations from the International Clinical Trials Registry Platform. The median trial sample size was 66. Conclusions We present an automated system for finding and categorizing RCTs. This yields a novel resource: a database of structured information automatically extracted for all published RCTs in humans. We make daily updates of this database available on our website (https://trialstreamer.robotreviewer.net).

Background and objectives The Cochrane Central Register of Controlled Trials (CENTRAL) is compiled from a number of sources, including PubMed and Embase. Since 2017, we have increased the number of sources feeding into CENTRAL and improved the efficiency of our processes through the use of application programming interfaces, machine learning, and crowdsourcing.Our objectives were twofold: (1) Assess the effectiveness of Cochrane's centralized search and screening processes to correctly identify references to published reports which are eligible for inclusion in Cochrane systematic reviews of randomized controlled trials (RCTs). (2) Identify opportunities to improve the performance of Cochrane's centralized search and screening processes to identify references to eligible trials. Methods We identified all references to RCTs (either published journal articles or trial registration records) with a publication or registration date between 1st January 2017 and 31st December 2018 that had been included in a Cochrane intervention review. We then viewed an audit trail for each included reference to determine if it had been identified by our centralized search process and subsequently added to CENTRAL. Results We identified 650 references to included studies with a publication year of 2017 or 2018. Of those, 634 (97.5%) had been captured by Cochrane's Centralised Search Service. Sixteen references had been missed by the Cochrane's Centralised Search Service: six had PubMed-not-MEDLINE status, four were missed by the centralized Embase search, three had been misclassified by Cochrane Crowd, one was from a journal not indexed in MEDLINE or Embase, one had only been added to Embase in 2019, and one reference had been rejected by the automated RCT machine learning classifier. Of the sixteen missed references, eight were the main or only publication to the trial in the review in which it had been included. Conclusion This analysis has shown that Cochrane's centralized search and screening processes are highly sensitive. It has also helped us to understand better why some references to eligible RCTs have been missed. The CSS is playing a critical role in helping to populate CENTRAL and is moving us toward making CENTRAL a comprehensive repository of RCTs.

The dynamics of gravitating astrophysical systems such as black holes and neutron stars are fascinatingly complex, offer some of nature's most spectacular phenomena, and capture the public's imagination in ways that few subjects can. Here, we describe {\it AstroDance}, a multi-media project to engage deaf and hard-of-hearing (DHH) students in astronomy and gravitational physics. {\it AstroDance} incorporates multiple means of representation of scientific concepts and was performed primarily for secondary and post-secondary audiences at $\sim$20 venues in the northeastern US prior to the historic first detection of gravitational waves. As part of the {\it AstroDance} project, we surveyed $\sim$1000 audience members roughly split evenly between hearing and DHH audience members. While both groups reported statistically equivalent high-rates of enjoyment of the performance, the DHH group reported an increase in how much they learned about science at a statistically significant rate compared to the hearing audience. Our findings suggest that multi-sensory approaches benefit both hearing and deaf audiences and enable accessible participation for broader groups., Comment: Accepted to Journal of Science Education for Students with Disabilities

Summary Owing to spatial segregation of tumor subclones, solid tumor sampling using formalin-fixed, paraffin-embedded blocks is often inadequate to represent the genomic heterogeneity of solid tumors. We present an approach, representative sampling, to dissect and homogenize leftover residual surgical tissue prior to sequencing. We also detail optional tumor cell enrichment and DNA preparation. This method, applicable only to surgically removed tumors with leftover tissue, facilitates robust sampling to avoid missing or over-representing actionable variants. For complete details on the use and execution of this protocol, please refer to Litchfield et al. (2020)., Graphical abstract, Highlights • Leftover surgical tumor tissue from patients with cancer is collected for sampling • This tissue can be dissected and homogenized to create a representative tumor sample • Multiple tumor types are amenable to dissection and homogenization • NGS is more accurate and can be further improved by tumor enrichment, Owing to spatial segregation of tumor subclones, solid tumor sampling using formalin-fixed, paraffin-embedded blocks is often inadequate to represent the genomic heterogeneity of solid tumors. We present an approach, representative sampling, to dissect and homogenize leftover residual surgical tissue prior to sequencing. We also detail optional tumor cell enrichment and DNA preparation. This method, applicable only to surgically removed tumors with leftover tissue, facilitates robust sampling to avoid missing or over-representing actionable variants.

New approaches to evidence synthesis, which use human effort and machine automation in mutually reinforcing ways, can enhance the feasibility and sustainability of living systematic reviews. Human effort is a scarce and valuable resource, required when automation is impossible or undesirable, and includes contributions from online communities (“crowds”) as well as more conventional contributions from review authors and information specialists. Automation can assist with some systematic review tasks, including searching, eligibility assessment, identification and retrieval of full-text reports, extraction of data, and risk of bias assessment. Workflows can be developed in which human effort and machine automation can each enable the other to operate in more effective and efficient ways, offering substantial enhancement to the productivity of systematic reviews. This paper describes and discusses the potential—and limitations—of new ways of undertaking specific tasks in living systematic reviews, identifying areas where these human/machine “technologies” are already in use, and where further research and development is needed. While the context is living systematic reviews, many of these enabling technologies apply equally to standard approaches to systematic reviewing.

Objectives Identifying all published reports of randomized controlled trials (RCTs) is an important aim, but it requires extensive manual effort to separate RCTs from non-RCTs, even using current machine learning (ML) approaches. We aimed to make this process more efficient via a hybrid approach using both crowdsourcing and ML. Methods We trained a classifier to discriminate between citations that describe RCTs and those that do not. We then adopted a simple strategy of automatically excluding citations deemed very unlikely to be RCTs by the classifier and deferring to crowdworkers otherwise. Results Combining ML and crowdsourcing provides a highly sensitive RCT identification strategy (our estimates suggest 95%–99% recall) with substantially less effort (we observed a reduction of around 60%–80%) than relying on manual screening alone. Conclusions Hybrid crowd-ML strategies warrant further exploration for biomedical curation/annotation tasks.

Background:\ud \ud The diagnosis of Alzheimer's disease dementia and other dementias relies on clinical assessment. There is a high prevalence of cognitive disorders, including undiagnosed dementia in secondary care settings. Short cognitive tests can be helpful in identifying those who require further specialist diagnostic assessment; however, there is a lack of consensus around the optimal tools to use in clinical practice. The Mini‐Cog is a short cognitive test comprising three‐item recall and a clock‐drawing test that is used in secondary care settings.\ud Objectives:\ud \ud The primary objective was to determine the diagnostic accuracy of the Mini‐Cog for detecting Alzheimer's disease dementia and other dementias in a secondary care setting. The secondary objectives were to investigate the heterogeneity of test accuracy in the included studies and potential sources of heterogeneity. These potential sources of heterogeneity will include the baseline prevalence of dementia in study samples, thresholds used to determine positive test results, the type of dementia (Alzheimer's disease dementia or all causes of dementia), and aspects of study design related to study quality.\ud Search methods:\ud \ud We searched the following sources in September 2012, with an update to 12 March 2019: Cochrane Dementia Group Register of Diagnostic Test Accuracy Studies, MEDLINE (OvidSP), Embase (OvidSP), BIOSIS Previews (Web of Knowledge), Science Citation Index (ISI Web of Knowledge), PsycINFO (OvidSP), and LILACS (BIREME). We made no exclusions with regard to language of Mini‐Cog administration or language of publication, using translation services where necessary.\ud Selection criteria:\ud \ud We included cross‐sectional studies and excluded case‐control designs, due to the risk of bias. We selected those studies that included the Mini‐Cog as an index test to diagnose dementia where dementia diagnosis was confirmed with reference standard clinical assessment using standardised dementia diagnostic criteria. We only included studies in secondary care settings (including inpatient and outpatient hospital participants).\ud Data collection and analysis:\ud \ud We screened all titles and abstracts generated by the electronic database searches. Two review authors independently checked full papers for eligibility and extracted data. We determined quality assessment (risk of bias and applicability) using the QUADAS‐2 tool. We extracted data into two‐by‐two tables to allow calculation of accuracy metrics for individual studies, reporting the sensitivity, specificity, and 95% confidence intervals of these measures, summarising them graphically using forest plots.\ud Main results:\ud \ud Three studies with a total of 2560 participants fulfilled the inclusion criteria, set in neuropsychology outpatient referrals, outpatients attending a general medicine clinic, and referrals to a memory clinic. Only n = 1415 (55.3%) of participants were included in the analysis to inform evaluation of Mini‐Cog test accuracy, due to the selective use of available data by study authors. There were concerns related to high risk of bias with respect to patient selection, and unclear risk of bias and high concerns related to index test conduct and applicability. In all studies, the Mini‐Cog was retrospectively derived from historic data sets. No studies included acute general hospital inpatients. The prevalence of dementia ranged from 32.2% to 87.3%. The sensitivities of the Mini‐Cog in the individual studies were reported as 0.67 (95% confidence interval (CI) 0.63 to 0.71), 0.60 (95% CI 0.48 to 0.72), and 0.87 (95% CI 0.83 to 0.90). The specificity of the Mini‐Cog for each individual study was 0.87 (95% CI 0.81 to 0.92), 0.65 (95% CI 0.57 to 0.73), and 1.00 (95% CI 0.94 to 1.00). We did not perform meta‐analysis due to concerns related to risk of bias and heterogeneity.\ud Authors' conclusions:\ud \ud This review identified only a limited number of diagnostic test accuracy studies using Mini‐Cog in secondary care settings. Those identified were at high risk of bias related to patient selection and high concerns related to index test conduct and applicability. The evidence was indirect, as all studies evaluated Mini‐Cog differently from the review question, where it was anticipated that studies would conduct Mini‐Cog and independently but contemporaneously perform a reference standard assessment to diagnose dementia. The pattern of test accuracy varied across the three studies. Future research should evaluate Mini‐Cog as a test in itself, rather than derived from other neuropsychological assessments. There is also a need for evaluation of the feasibility of the Mini‐Cog for the diagnosis of dementia to help adequately determine its role in the clinical pathway.

Background:\ud Dementia assessment often involves initial screening, using a brief tool, followed by more detailed assessment where required. The AD‐8 is a short questionnaire, completed by a suitable 'informant' who knows the person well. AD‐8 is designed to assess change in functional performance secondary to cognitive change.\ud \ud Objectives:\ud To determine the diagnostic accuracy of the informant‐based AD‐8 questionnaire, in detection of all‐cause (undifferentiated) dementia in adults. Where data were available, we described the following: the diagnostic accuracy of the AD‐8 at various predefined threshold scores; the diagnostic accuracy of the AD‐8 for each healthcare setting and the effects of heterogeneity on the reported diagnostic accuracy of the AD‐8.\ud \ud Search Methods:\ud We searched the following sources on 27 May 2014, with an update to 7 June 2018: ALOIS (Cochrane Dementia and Cognitive Improvement Group), MEDLINE (Ovid SP), Embase (Ovid SP), PsycINFO (Ovid SP), BIOSIS Previews (Thomson Reuters Web of Science), Web of Science Core Collection (includes Conference Proceedings Citation Index) (Thomson Reuters Web of Science), CINAHL (EBSCOhost) and LILACS (BIREME). We checked reference lists of relevant studies and reviews, used searches of known relevant studies in PubMed to track related articles, and contacted research groups conducting work on the AD‐8 to try to find additional studies. We developed a sensitive search strategy and used standardised database subject headings as appropriate. Foreign language publications were translated.\ud \ud Selection Criteria:\ud We selected those studies which included the AD‐8 to assess for the presence of dementia and where dementia diagnosis was confirmed with clinical assessment. We only included those studies where the AD‐8 was used as an informant assessment. We made no exclusions in relation to healthcare setting, language of AD‐8 or the AD‐8 score used to define a 'test positive' case.\ud \ud Data Collection and Analysis:\ud We screened all titles generated by electronic database searches, and reviewed abstracts of potentially relevant studies. Two independent assessors checked full papers for eligibility and extracted data. We extracted data into two‐by‐two tables to allow calculation of accuracy metrics for individual studies. We then created summary estimates of sensitivity, specificity and likelihood ratios using the bivariate approach and plotting results in receiver operating characteristic (ROC) space. We determined quality assessment (risk of bias and applicability) using the QUADAS‐2 tool.\ud \ud Main Results:\ud From 36 papers describing AD‐8 test accuracy, we included 10 papers. We utilised data from nine papers with 4045 individuals, 1107 of whom (27%) had a clinical diagnosis of dementia. Pooled analysis of seven studies, using an AD‐8 informant cut‐off score of two, indicated that sensitivity was 0.92 (95% confidence interval (CI) 0.86 to 0.96); specificity was 0.64 (95% CI 0.39 to 0.82); the positive likelihood ratio was 2.53 (95% CI 1.38 to 4.64); and the negative likelihood ratio was 0.12 (95% CI 0.07 to 0.21). Pooled analysis of five studies, using an AD‐8 informant cut‐off score of three, indicated that sensitivity was 0.91 (95% CI 0.80 to 0.96); specificity was 0.76 (95% CI 0.57 to 0.89); the positive likelihood ratio was 3.86 (95% CI 2.03 to 7.34); and the negative likelihood ratio was 0.12 (95% CI 0.06 to 0.24).\ud \ud Four studies were conducted in community settings; four were in secondary care (one in the acute hospital); and one study was in primary care. The AD‐8 has a higher relative sensitivity (1.11, 95% CI 1.02 to 1.21), but lower relative specificity (0.51, 95% CI 0.23 to 1.09) in secondary care compared to community care settings.\ud \ud There was heterogeneity across the included studies. Dementia prevalence rate varied from 12% to 90% of included participants. The tool was also used in various different languages. Among all the included studies there was evidence of risk of bias. Issues included the selection of participants, conduct of index test, and flow of assessment procedures.\ud \ud Authors' Conclusions:\ud The high sensitivity of the AD‐8 suggests it can be used to identify adults who may benefit from further specialist assessment and diagnosis, but is not a diagnostic test in itself. This pattern of high sensitivity and lower specificity is often suited to a screening test. Test accuracy varies by setting, however data in primary care and acute hospital settings are limited. This review identified significant heterogeneity and risk of bias, which may affect the validity of its summary findings.

The results of a survey representing the reactions and responses of 29,095 individuals across the USA who witnessed the eclipse or participated in eclipse-related events on or around August 21, 2017, are reported. The surveys collect a body of data showing the demographics of these respondents and an analysis of the responses they gave in terms of motivations for participation, learning, future plans as a result of the event, and in summarizing the overall experience. Conclusions show future efforts at similar events could be directed to broaden participation and increase engagement.

© 2014 The Cochrane Collaboration. This is the protocol for a review and there is no abstract. The objectives are as follows: To determine the diagnostic accuracy of the plasma and CSF Abeta42 index tests for distinguishing Alzheimer's disease dementia from each of the other forms of dementia in people who meet the general criteria for a dementia syndrome. To investigate the heterogeneity of test accuracy in the included studies. We expect that heterogeneity will be likely and that it will be an important component of the review. The potential sources of heterogeneity, which will be used as a framework for the investigation of heterogeneity, include target population, index test, target disorder and study quality and are detailed in the analysis section.

This study was supported by Cancer Research UK, AstraZeneca and Pfizer UK. Introduction: Phase I of the Cancer Research UK Stratified Medicine Programme (SMP1) was designed to roll out molecular pathology testing nationwide at the point of cancer diagnosis, as well as facilitate an infrastructure where surplus cancer tissue could be used for research. It offered a non-trial setting to examine common UK cancer genetics in a real-world context. Methods: A total of 26 sites in England, Wales and Scotland, recruited samples from 7814 patients for genetic examination between 2011 and 2013. Tumour types involved were breast, colorectal, lung, prostate, ovarian cancer and malignant melanoma. Centralised molecular testing of surplus material from resections or biopsies of primary/metastatic tissue was performed, with samples examined for 3-5 genetic alterations deemed to be of key interest in site-specific cancers by the National Cancer Research Institute Clinical Study groups. Results: 10 754 patients (98% of those approached) consented to participate, from which 7814 tumour samples were genetically analysed. In total, 53% had at least one genetic aberration detected. From 1885 patients with lung cancer, KRAS mutation was noted to be highly prevalent in adenocarcinoma (37%). In breast cancer (1873 patients), there was a striking contrast in TP53 mutation incidence between patients with ductal cancer (27.3%) and lobular cancer (3.4%). Vast inter-tumour heterogeneity of colorectal cancer (1550 patients) was observed, including myriad double and triple combinations of genetic aberrations. Significant losses of important clinical information included smoking status in lung cancer and loss of distinction between low-grade and high-grade serous ovarian cancers. Conclusion: Nationwide molecular pathology testing in a non-trial setting is feasible. The experience with SMP1 has been used to inform ongoing CRUK flagship programmes such as the CRUK National Lung MATRIX trial and TRACERx. Publisher PDF

ObjectiveTo determine whether methodological and reporting quality are associated with surrogate measures of publication impact in the field of dementia biomarker studies.MethodsWe assessed dementia biomarker studies included in a previous systematic review in terms of methodological and reporting quality using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) and Standards for Reporting of Diagnostic Accuracy (STARD), respectively. We extracted additional study and journal-related data from each publication to account for factors shown to be associated with impact in previous research. We explored associations between potential determinants and measures of publication impact in univariable and stepwise multivariable linear regression analyses.Outcome measuresWe aimed to collect data on four measures of publication impact: two traditional measures—average number of citations per year and 5-year impact factor of the publishing journal and two alternative measures—the Altmetric Attention Score and counts of electronic downloads.ResultsThe systematic review included 142 studies. Due to limited data, Altmetric Attention Scores and electronic downloads were excluded from the analysis, leaving traditional metrics as the only analysed outcome measures. We found no relationship between QUADAS and traditional metrics. Citation rates were independently associated with 5-year journal impact factor (β=0.42; pConclusionsCitation rates and 5-year journal impact factor appear to measure different dimensions of impact. Citation rates were weakly associated with completeness of reporting, while neither traditional metric was related to methodological rigour. Our results suggest that high publication usage and journal outlet is not a guarantee of quality and readers should critically appraise all papers regardless of presumed impact.

Objective Biomarkers are of increasing interest in dementia research. Studies describing favourable accuracy of various dementia tests have influenced research, guidelines and diagnostic criteria. Publication bias is known to compromise reports on efficacy of therapeutic interventions. Traditional methods of quantifying publication bias are not suited to reviews of diagnostic tests. We aimed to describe rates and predictors of publication of dementia test accuracy studies presented at scientific meetings. Methods We chose three exemplar scientific meetings from 2009. Two independent researchers assessed conference proceedings and selected all abstracts relating to dementia diagnostics. We recorded basic descriptors and dichotomised results as ‘positive’ or ‘neutral’. We assessed publication status using electronic literature databases and contacting lead authors. We described univariate and multivariate predictors of publication status using logistic regression modelling. Results From n = 2257 abstracts, we identified n = 250 (11%) abstracts relating to dementia diagnostics. The majority n = 209 (84%) reported positive results. Only 97 (39%) of these studies are published. Univariate predictors of publication status included positive result (p = 0.042), North American or European authors (p = 0.047), higher number of participants (p = 0.008) and use of a ‘biomarker’ test (p = 0.035). On multivariate analysis, only increasing number of participants was independently associated with publication (p = 0.034). Conclusions Our strategy did not prove or disprove a publication bias effect in dementia test accuracy studies. The substantial proportion of ‘positive’ studies may point to a downstream ‘submission bias’ effect on decision to submit data to meetings. Modest rate of publication of dementia test accuracy studies is concerning, and publication bias remains possible. Copyright © 2015 John Wiley & Sons, Ltd.

We present Chandra and XMM-Newton X-ray, VLA radio, and optical observations of three candidate Compact Steep Spectrum (CSS) radio galaxies. CSS sources are galactic scale and are presumably driving a shock through the ISM of their host galaxy. B3 1445+410 is a low excitation emission line CSS radio galaxy with possibly a hybrid Fanaroff-Riley FRI/II (or Fat Double) radio morphology. The Chandra observations reveal a point-like source which is well fit with a power law consistent with emission from a Doppler boosted core. 3C 268.3 is a CSS broad line radio galaxy whose Chandra data are consistent spatially with a point source centered on the nucleus and spectrally with a double power-law model. PKS B1017-325 is a low excitation emission line radio galaxy with a bent double radio morphology. While from our new spectroscopic redshift, PKS B1017-325 falls outside the formal definition of a CSS, the XMNM-Newton observations are consistent with ISM emission with either a contribution from hot shocked gas or non-thermal jet emission. We compile selected radio and X-ray properties of the nine bona fide CSS radio galaxies with X-ray detections so far. We find that 2/9 show X-ray spectroscopic evidence for hot shocked gas. We note that the counts in the sources are low and the properties of the 2 sources with evidence for hot shocked gas are typical of the other CSS radio galaxies. We suggest that hot shocked gas may be typical of CSS radio galaxies due to their propagation through their host galaxies., 15 pages, 12 figures. Accepted for publication in the Astrophysical Journal

While it is important for the evidence supporting practice guidelines to be current, that is often not the case. The advent of living systematic reviews has made the concept of "living guidelines" realistic, with the promise to provide timely, up-to-date and high-quality guidance to target users. We define living guidelines as an optimization of the guideline development process to allow updating individual recommendations as soon as new relevant evidence becomes available. A major implication of that definition is that the unit of update is the individual recommendation and not the whole guideline. We then discuss when living guidelines are appropriate, the workflows required to support them, the collaboration between living systematic reviews and living guideline teams, the thresholds for changing recommendations, and potential approaches to publication and dissemination. The success and sustainability of the concept of living guideline will depend on those of its major pillar, the living systematic review. We conclude that guideline developers should both experiment with and research the process of living guidelines.

[This corrects the article DOI: 10.1186/s41512-016-0001-y.].

Machine learning (ML) algorithms have proven highly accurate for identifying Randomized Controlled Trials (RCTs) but are not used much in practice, in part because the best way to make use of the technology in a typical workflow is unclear. In this work, we evaluate ML models for RCT classification (support vector machines, convolutional neural networks, and ensemble approaches). We trained and optimized support vector machine and convolutional neural network models on the titles and abstracts of the Cochrane Crowd RCT set. We evaluated the models on an external dataset (Clinical Hedges), allowing direct comparison with traditional database search filters. We estimated area under receiver operating characteristics (AUROC) using the Clinical Hedges dataset. We demonstrate that ML approaches better discriminate between RCTs and non-RCTs than widely used traditional database search filters at all sensitivity levels; our best-performing model also achieved the best results to date for ML in this task (AUROC 0.987, 95% CI, 0.984–0.989). We provide practical guidance on the role of ML in (1) systematic reviews (high-sensitivity strategies) and (2) rapid reviews and clinical question answering (high-precision strategies) together with recommended probability cutoffs for each use case. Finally, we provide open-source software to enable these approaches to be used in practice.

BackgroundIn primary care, general practitioners (GPs) unavoidably reach a clinical judgement about a patient as part of their encounter with patients, and so clinical judgement can be an important part of the diagnostic evaluation. Typically clinical decision making about what to do next for a patient incorporates clinical judgement about the diagnosis with severity of symptoms and patient factors, such as their ideas and expectations for treatment. When evaluating patients for dementia, many GPs report using their own judgement to evaluate cognition, using information that is immediately available at the point of care, to decide whether someone has or does not have dementia, rather than more formal tests.ObjectivesTo determine the diagnostic accuracy of GPs’ clinical judgement for diagnosing cognitive impairment and dementia in symptomatic people presenting to primary care. To investigate the heterogeneity of test accuracy in the included studies.Search methodsWe searched MEDLINE (Ovid SP), Embase (Ovid SP), PsycINFO (Ovid SP), Web of Science Core Collection (ISI Web of Science), and LILACs (BIREME) on 16 September 2021.Selection criteriaWe selected cross‐sectional and cohort studies from primary care where clinical judgement was determined by a GP either prospectively (after consulting with a patient who has presented to a specific encounter with the doctor) or retrospectively (based on knowledge of the patient and review of the medical notes, but not relating to a specific encounter with the patient). The target conditions were dementia and cognitive impairment (mild cognitive impairment and dementia) and we included studies with any appropriate reference standard such as the Diagnostic and Statistical Manual of Mental Disorders (DSM), International Classification of Diseases (ICD), aetiological definitions, or expert clinical diagnosis.Data collection and analysisTwo review authors screened titles and abstracts for relevant articles and extracted data separately with differences resolved by consensus discussion. We used QUADAS‐2 to evaluate the risk of bias and concerns about applicability in each study using anchoring statements. We performed meta‐analysis using the bivariate method.Main resultsWe identified 18,202 potentially relevant articles, of which 12,427 remained after de‐duplication. We assessed 57 full‐text articles and extracted data on 11 studies (17 papers), of which 10 studies had quantitative data. We included eight studies in the meta‐analysis for the target condition dementia and four studies for the target condition cognitive impairment. Most studies were at low risk of bias as assessed with the QUADAS‐2 tool, except for the flow and timing domain where four studies were at high risk of bias, and the reference standard domain where two studies were at high risk of bias. Most studies had low concern about applicability to the review question in all QUADAS‐2 domains.Average age ranged from 73 years to 83 years (weighted average 77 years). The percentage of female participants in studies ranged from 47% to 100%. The percentage of people with a final diagnosis of dementia was between 2% and 56% across studies (a weighted average of 21%). For the target condition dementia, in individual studies sensitivity ranged from 34% to 91% and specificity ranged from 58% to 99%. In the meta‐analysis for dementia as the target condition, in eight studies in which a total of 826 of 2790 participants had dementia, the summary diagnostic accuracy of clinical judgement of general practitioners was sensitivity 58% (95% confidence interval (CI) 43% to 72%), specificity 89% (95% CI 79% to 95%), positive likelihood ratio 5.3 (95% CI 2.4 to 8.2), and negative likelihood ratio 0.47 (95% CI 0.33 to 0.61).For the target condition cognitive impairment, in individual studies sensitivity ranged from 58% to 97% and specificity ranged from 40% to 88%. The summary diagnostic accuracy of clinical judgement of general practitioners in four studies in which a total of 594 of 1497 participants had cognitive impairment was sensitivity 84% (95% CI 60% to 95%), specificity 73% (95% CI 50% to 88%), positive likelihood ratio 3.1 (95% CI 1.4 to 4.7), and negative likelihood ratio 0.23 (95% CI 0.06 to 0.40).It was impossible to draw firm conclusions in the analysis of heterogeneity because there were small numbers of studies. For specificity we found the data were compatible with studies that used ICD‐10, or applied retrospective judgement, had higher reported specificity compared to studies with DSM definitions or using prospective judgement. In contrast for sensitivity, we found studies that used a prospective index test may have had higher sensitivity than studies that used a retrospective index test.Authors' conclusionsClinical judgement of GPs is more specific than sensitive for the diagnosis of dementia. It would be necessary to use additional tests to confirm the diagnosis for either target condition, or to confirm the absence of the target conditions, but clinical judgement may inform the choice of further testing. Many people who a GP judges as having dementia will have the condition. People with false negative diagnoses are likely to have less severe disease and some could be identified by using more formal testing in people who GPs judge as not having dementia. Some false positives may require similar practical support to those with dementia, but some ‐ such as some people with depression ‐ may suffer delayed intervention for an alternative treatable pathology.

This invited editorial describes the achievements of the last 21 years of the Cochrane Dementia and Cognitive Improvement Group (DR Quinn is the coordinating editor of the group).

A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

Systematic reviews are difficult to keep up to date, but failure to do so leads to a decay in review currency, accuracy, and utility. We are developing a novel approach to systematic review updating termed "Living systematic review" (LSR): systematic reviews that are continually updated, incorporating relevant new evidence as it becomes available. LSRs may be particularly important in fields where research evidence is emerging rapidly, current evidence is uncertain, and new research may change policy or practice decisions. We hypothesize that a continual approach to updating will achieve greater currency and validity, and increase the benefits to end users, with feasible resource requirements over time.

unKnown

Background Intratumour heterogeneity is recognised across different tumour types and has implications for therapeutic resistance. At present, clinical practice often relies upon molecular information derived from a single biopsy of a primary or metastatic tumour. This information guides treatment choice but may not be representative of the diversity of the tumour. It is currently difficult to evaluate how effectively a single region guides treatment decisions because the formalin-fixed residual surgical sample that is not paraffin embedded for diagnostic purposes is typically thrown away. Retention and homogenisation – ‘blending’- of this residual formalin-fixed leftover tumour tissue creates a more representative sample for analysis. DNA may be extracted from this sample for sequencing. Pilot data in kidney cancer has demonstrated the potential of this methodology for robust mutational calling, accurate determination of cancer cell fraction and the ability to discern clonal from subclonal variants. Such information may be clinically relevant; for example, discerning resistant subclones prior to treatment, or identifying clonal neoantigens worth targeting with immunotherapy. Trial design In order to establish the feasibility of homogenization as a potential companion diagnostic tool, our study aims to 1) identify the proportion of primary tumour cases that have left over tissue amenable to homogenization across multiple tumour types and 2) pilot homogenization across multiple tumour types. The molecular profile of the homogenate will be compared to that obtained from the diagnostic specimen using next generation sequencing techniques. This is a prospective non-interventional study (NCT03832062). Patients undergoing surgical intervention at The Royal Marsden Hospital (NHS Foundation Trust) with leftover tumour tissue from primary breast, colorectal, gastric, pancreatic, ovarian, renal cancer and sarcoma surgeries, as well as melanoma lymph node dissections will be included in the feasibility assessment. We plan to homogenise 500 cases across different tumour types. The study opened in September 2018 and is expected to run for 2 years. Clinical trial identification NCT03832062; Release date: February 2019. Legal entity responsible for the study Royal Marsden NHS Foundation Trust. Funding Ventana Medical Systems (a subsidiary of Roche). Disclosure L. Gallegos: Full / Part-time employment: Roche. S. Hill: Full / Part-time employment: Roche. A. Barhoumi: Full / Part-time employment: Roche. S. Stanislaw: Full / Part-time employment: Roche. M. Mendoza: Full / Part-time employment: Roche. J.M.G. Larkin: Research grant / Funding (institution): Roche; Advisory / Consultancy: Roche. N. Alexander: Full / Part-time employment: Roche; Shareholder / Stockholder / Stock options: Roche. S. Turajlic: Research grant / Funding (institution): Ventana Medical Systems (subsidiary of Roche). All other authors have declared no conflicts of interest.

BackgroundIn a research study, to give a comprehensive evaluation of the impact of interventions, the outcome measures should reflect the lived experience of the condition. In dementia studies, this necessitates the use of outcome measures which capture the range of disease effects, not limited to cognitive functioning. In particular, assessing the functional impact of cognitive impairment is recommended by regulatory authorities, but there is no consensus on the optimal approach for outcome assessment in dementia research. Our aim was to describe the outcome measures used in dementia and mild cognitive impairment (MCI) intervention studies, with particular interest in those evaluating patient-centred outcomes of functional performance and quality of life.MethodsWe performed a focused review of the literature with multiple embedded checks of internal and external validity. We used the Cochrane Dementia and Cognitive Improvement Group’s register of dementia studies, ALOIS. ALOIS was searched to obtain records of all registered dementia and MCI intervention studies over a 10-year period (2004–2014). We included both published and unpublished materials. Outcomes were categorised as cognitive, functional, quality of life, mood, behaviour, global/disease severity and institutionalisation.ResultsFrom an initial return of 3271 records, we included a total of 805 records, including 676 dementia trial records and 129 MCI trial records. Of these, 78 % (630) originated from peer-reviewed publications and 60 % (487) reported results of pharmacological interventions. Cognitive outcomes were reported in 70 % (563), in contrast with 29 % (237) reporting measures of functional performance and only 13 % (102) reporting quality of life measures. We identified significant heterogeneity in the tools used to capture these outcomes, with frequent use of non-standardised tests.ConclusionsThis focus on cognitive performance questions the extent to which intervention studies for dementia are evaluating outcome measures which are relevant to individual patients and their carers. The heterogeneity in measures, use of bespoke tools and poor descriptions of test strategy all support the need for a more standardised approach to the conduct and reporting of outcomes assessments.

Background: \ud The Informant Questionnaire for Cognitive Decline in the Elderly (IQCODE) is a structured interview based on informant responses that is used to assess for possible dementia. IQCODE has been used for retrospective or contemporaneous assessment of cognitive decline. There is considerable interest in tests that may identify those at future risk of developing dementia. Assessing a population free of dementia for the prospective development of dementia is an approach often used in studies of dementia biomarkers. In theory, questionnaire-based assessments, such as IQCODE, could be used in a similar way, assessing for dementia that is diagnosed on a later (delayed) assessment.\ud \ud Objectives: \ud To determine the diagnostic accuracy of IQCODE in a population free from dementia for the delayed diagnosis of dementia (test accuracy with delayed verification study design).\ud \ud Search methods: \ud We searched these sources on 16 January 2016: ALOIS (Cochrane Dementia and Cognitive Improvement Group), MEDLINE Ovid SP, Embase Ovid SP, PsycINFO Ovid SP, BIOSIS Previews on Thomson Reuters Web of Science, Web of Science Core Collection (includes Conference Proceedings Citation Index) on Thomson Reuters Web of Science, CINAHL EBSCOhost, and LILACS BIREME. We also searched sources specific to diagnostic test accuracy: MEDION (Universities of Maastricht and Leuven); DARE (Database of Abstracts of Reviews of Effects, in the Cochrane Library); HTA Database (Health Technology Assessment Database, in the Cochrane Library), and ARIF (Birmingham University). We checked reference lists of included studies and reviews, used searches of included studies in PubMed to track related articles, and contacted research groups conducting work on IQCODE for dementia diagnosis to try to find additional studies. We developed a sensitive search strategy; search terms were designed to cover key concepts using several different approaches run in parallel, and included terms relating to cognitive tests, cognitive screening, and dementia. We used standardised database subject headings, such as MeSH terms (in MEDLINE) and other standardised headings (controlled vocabulary) in other databases, as appropriate.\ud \ud Selection criteria: \ud We selected studies that included a population free from dementia at baseline, who were assessed with the IQCODE and subsequently assessed for the development of dementia over time. The implication was that at the time of testing, the individual had a cognitive problem sufficient to result in an abnormal IQCODE score (defined by the study authors), but not yet meeting dementia diagnostic criteria.\ud \ud Data collection and analysis: \ud We screened all titles generated by the electronic database searches, and reviewed abstracts of all potentially relevant studies. Two assessors independently checked the full papers for eligibility and extracted data. We determined quality assessment (risk of bias and applicability) using the QUADAS-2 tool, and reported quality using the STARDdem tool.\ud \ud Main results: \ud From 85 papers describing IQCODE, we included three papers, representing data from 626 individuals. Of this total, 22% (N = 135/626) were excluded because of prevalent dementia. There was substantial attrition; 47% (N = 295) of the study population received reference standard assessment at first follow-up (three to six months) and 28% (N = 174) received reference standard assessment at final follow-up (one to three years). Prevalence of dementia ranged from 12% to 26% at first follow-up and 16% to 35% at final follow-up.\ud \ud The three studies were considered to be too heterogenous to combine, so we did not perform meta-analyses to describe summary estimates of interest. Included patients were poststroke (two papers) and hip fracture (one paper). The IQCODE was used at three thresholds of positivity (higher than 3.0, higher than 3.12 and higher than 3.3) to predict those at risk of a future diagnosis of dementia. Using a cut-off of 3.0, IQCODE had a sensitivity of 0.75 (95%CI 0.51 to 0.91) and a specificity of 0.46 (95%CI 0.34 to 0.59) at one year following stroke. Using a cut-off of 3.12, the IQCODE had a sensitivity of 0.80 (95%CI 0.44 to 0.97) and specificity of 0.53 (95C%CI 0.41 to 0.65) for the clinical diagnosis of dementia at six months after hip fracture. Using a cut-off of 3.3, the IQCODE had a sensitivity of 0.84 (95%CI 0.68 to 0.94) and a specificity of 0.87 (95%CI 0.76 to 0.94) for the clinical diagnosis of dementia at one year after stroke.\ud \ud In generaI, the IQCODE was sensitive for identification of those who would develop dementia, but lacked specificity. Methods for both excluding prevalent dementia at baseline and assessing for the development of dementia were varied, and had the potential to introduce bias.\ud \ud Authors' conclusions: \ud Included studies were heterogenous, recruited from specialist settings, and had potential biases. The studies identified did not allow us to make specific recommendations on the use of the IQCODE for the future diagnosis of dementia in clinical practice. The included studies highlighted the challenges of delayed verification dementia research, with issues around prevalent dementia assessment, loss to follow-up over time, and test non-completion potentially limiting the studies. Future research should recognise these issues and have explicit protocols for dealing with them.

Background In a research study, to give a comprehensive evaluation of the impact of interventions, the outcome measures should reflect the lived experience of the condition. In dementia studies, this necessitates the use of outcome measures which capture the range of disease effects, not limited to cognitive functioning. In particular, assessing the functional impact of cognitive impairment is recommended by regulatory authorities, but there is no consensus on the optimal approach for outcome assessment in dementia research. Our aim was to describe the outcome measures used in dementia and mild cognitive impairment (MCI) intervention studies, with particular interest in those evaluating patient-centred outcomes of functional performance and quality of life. Methods We performed a focused review of the literature with multiple embedded checks of internal and external validity. We used the Cochrane Dementia and Cognitive Improvement Group’s register of dementia studies, ALOIS. ALOIS was searched to obtain records of all registered dementia and MCI intervention studies over a 10-year period (2004–2014). We included both published and unpublished materials. Outcomes were categorised as cognitive, functional, quality of life, mood, behaviour, global/disease severity and institutionalisation. Results From an initial return of 3271 records, we included a total of 805 records, including 676 dementia trial records and 129 MCI trial records. Of these, 78 % (630) originated from peer-reviewed publications and 60 % (487) reported results of pharmacological interventions. Cognitive outcomes were reported in 70 % (563), in contrast with 29 % (237) reporting measures of functional performance and only 13 % (102) reporting quality of life measures. We identified significant heterogeneity in the tools used to capture these outcomes, with frequent use of non-standardised tests. Conclusions This focus on cognitive performance questions the extent to which intervention studies for dementia are evaluating outcome measures which are relevant to individual patients and their carers. The heterogeneity in measures, use of bespoke tools and poor descriptions of test strategy all support the need for a more standardised approach to the conduct and reporting of outcomes assessments. Electronic supplementary material The online version of this article (doi:10.1186/s13195-016-0216-8) contains supplementary material, which is available to authorized users.

Background: Multiple publication of clinical studies makes it difficult for researchers and the public to identify unique studies from reference lists and bibliographic searches. The Cochrane Dementia and Cognitive Improvement Group's Specialised Register has recently been extended to include reports of all trials of prevention strategies and cognitive enhancers, as well as of treatment and rehabilitation in dementia. It has also been made freely available on-line to the community of researchers and the public (http://www.medicine.ox.ac.uk/alois/). The objective of this project is to create and maintain a comprehensive, up-to-date study-based registry of all trials and to make this register freely available on-line. Methods: We developed highly sensitive search strategies for the retrieval of citations of studies of treatment, prevention, and cognitive enhancement. These were then run in major databases, trial registers and grey literature sources. Each RCT was read and data extracted. Duplicate publications were linked to the same trial record. Each record was linked to a published Cochrane review where appropriate. A user friendly, web interface for searching the database was created. Results: ALOIS contains records of 2,525 randomized controlled trials and 495 controlled clinical trials in dementia and cognitive impairment. As at 1st April 2009 this includes all completed, ongoing and aborted studies identified by sensitive monthly searches. As well as citations, the displayed data include: study design/blinding, participants and health condition, intervention and dosage, number of participants, outcomes, date of study, website links and related Cochrane reviews/protocols. Conclusions: ALOIS is an online study-based register which will supply review authors, investigators, researchers and other stakeholders with a unique and up-to-date source of all published and unpublished studies in the area of dementia treatment/prevention and cognitive enhancement.

BACKGROUND: Although recent diagnostic criteria for Alzheimer's disease propose the use of biomarkers, validation of these biomarkers by diagnostic test accuracy studies is a necessary first step, followed by the synthesis of the evidence from these studies in systematic reviews and meta-analyses. The quality of the resulting evidence depends on the number and size of the primary studies, their quality, and the adequacy of their reporting. This systematic review assesses the weight and quality of the evidence available from primary diagnostic test accuracy studies. METHODS: A MEDLINE search was performed in August 2011 to identify all potentially relevant publications relating to the biomarkers β-amyloid, tau, positron emission tomography ((18)F-fluorodeoxyglucose or ligands for amyloid), or magnetic resonance imaging (MRI). The reporting and methodology were assessed using the Standards for Reporting of Diagnostic Accuracy and Quality Assessment of Diagnostic Accuracy Studies assessment tools, respectively. Because clinical progression to dementia is the most commonly used reference standard, this review focuses on participants with objective cognitive impairment but no dementia at baseline. RESULTS: Of the 19,104 published references identified by the search, 142 longitudinal studies relating to the biomarkers of interest were identified, which included subjects who had objective cognitive impairment but no dementia at baseline. The highest number of studies (n = 70) and of participants (n = 4722) related to structural MRI. MRI also yielded the highest number of studies with extractable data for meta-analysis (n = 32 [46% of all structural MRI studies]), followed by cerebrospinal fluid tau (n = 24 [73%]). There were few studies on positron emission tomography ligands for amyloid having suitable data for meta-analysis (n = 4). There was considerable variation across studies in reporting outcomes, methods of blinding and selection, means of accounting for indeterminate or missing values, the interval between the test and assessments, and the determination of test thresholds. CONCLUSIONS: The body of evidence for biomarkers is not large and is variable across the different types of biomarkers. Important information is missing from many study reports, highlighting the need for standardization of methodology and reporting to improve the rigor of biomarker validation.