Search

Introduction Large language model (LLM)-linked chatbots are being increasingly applied in healthcare due to their impressive functionality and public availability. Studies have assessed the ability of LLM-linked chatbots to provide accurate clinical advice. However, the methods applied in these Chatbot Assessment Studies are inconsistent due to the lack of reporting standards available, which obscures the interpretation of their study findings. This protocol outlines the development of the Chatbot Assessment Reporting Tool (CHART) reporting guideline.Methods and analysis The development of the CHART reporting guideline will consist of three phases, led by the Steering Committee. During phase one, the team will identify relevant reporting guidelines with artificial intelligence extensions that are published or in development by searching preprint servers, protocol databases, and the Enhancing the Quality and Transparency of health research Network. During phase two, we will conduct a scoping review to identify studies that have addressed the performance of LLM-linked chatbots in summarising evidence and providing clinical advice. The Steering Committee will identify methodology used in previous Chatbot Assessment Studies. Finally, the study team will use checklist items from prior reporting guidelines and findings from the scoping review to develop a draft reporting checklist. We will then perform a Delphi consensus and host two synchronous consensus meetings with an international, multidisciplinary group of stakeholders to refine reporting checklist items and develop a flow diagram.Ethics and dissemination We will publish the final CHART reporting guideline in peer-reviewed journals and will present findings at peer-reviewed meetings. Ethical approval was submitted to the Hamilton Integrated Research Ethics Board and deemed “not required” in accordance with the Tri-Council Policy Statement (TCPS2) for the development of the CHART reporting guideline (#17025).Registration This study protocol is preregistered with Open Science Framework: https://doi.org/10.17605/OSF.IO/59E2Q.

OBJECTIVE: Back pain is common in the working population. This systematic review with network meta-analysis (NMA) aimed to compare the effects of interventions for preventing back pain among office workers. METHODS: We searched eight databases and additional sources up to March 2021. We included randomized controlled trials (RCT) and cluster RCT focusing on office workers, comparing work-related interventions aimed at preventing back pain (defined as pain in any part of the spine) to a control condition and assessing back pain and/or work absence. Further outcomes considered were adverse events and participants’ satisfaction. We performed both frequentist and component NMA. Risk of bias (RoB) was evaluated using RoB 2 and certainty of the evidence (CoE) was assessed using GRADE. RESULTS: We screened 9809 records and included 24 studies with a total of 7080 participants. RoB was assessed as “some concerns” or “high” for all studies and outcomes. Included studies investigated multicomponent interventions, ergonomics, physical activity, education, behavioral interventions and no/minimal interventions. Effects were mostly not statistically significant and based on low/very low CoE. Physical activity probably reduces days of work absence slightly [mean difference (MD) -1.10, 95% confidence interval (CI) -2.07– -0.13], and combining physical activity and ergonomics may reduce back pain intensity (standardized MD -0.41, 95% CI -0.80– -0.02) when compared to no/minimal intervention. A large proportion of participants were satisfied with the interventions, adverse events were rarely assessed. CONCLUSIONS: We observed mostly minor effects of interventions on back pain and work absence among office workers. The practical relevance of these effects is questionable.

BackgroundClinical practice guidelines are systematically developed statements intended to optimize patient care. However, a gapless implementation of guideline recommendations requires health care personnel not only to be aware of the recommendations and to support their content but also to recognize every situation in which they are applicable. To not miss situations in which recommendations should be applied, computerized clinical decision support can be provided through a system that allows an automated monitoring of adherence to clinical guideline recommendations in individual patients. ObjectiveThis study aims to collect and analyze the requirements for a system that allows the monitoring of adherence to evidence-based clinical guideline recommendations in individual patients and, based on these requirements, to design and implement a software prototype that integrates guideline recommendations with individual patient data, and to demonstrate the prototype’s utility in treatment recommendations. MethodsWe performed a work process analysis with experienced intensive care clinicians to develop a conceptual model of how to support guideline adherence monitoring in clinical routine and identified which steps in the model could be supported electronically. We then identified the core requirements of a software system to support recommendation adherence monitoring in a consensus-based requirements analysis within the loosely structured focus group work of key stakeholders (clinicians, guideline developers, health data engineers, and software developers). On the basis of these requirements, we designed and implemented a modular system architecture. To demonstrate its utility, we applied the prototype to monitor adherence to a COVID-19 treatment recommendation using clinical data from a large European university hospital. ResultsWe designed a system that integrates guideline recommendations with real-time clinical data to evaluate individual guideline recommendation adherence and developed a functional prototype. The needs analysis with clinical staff resulted in a flowchart describing the work process of how adherence to recommendations should be monitored. Four core requirements were identified: the ability to decide whether a recommendation is applicable and implemented for a specific patient, the ability to integrate clinical data from different data formats and data structures, the ability to display raw patient data, and the use of a Fast Healthcare Interoperability Resources–based format for the representation of clinical practice guidelines to provide an interoperable, standards-based guideline recommendation exchange format. ConclusionsOur system has advantages in terms of individual patient treatment and quality management in hospitals. However, further studies are needed to measure its impact on patient outcomes and evaluate its resource effectiveness in different clinical settings. We specified a modular software architecture that allows experts from different fields to work independently and focus on their area of expertise. We have released the source code of our system under an open-source license and invite for collaborative further development of the system.

Objective To examine the effects of different nutritional intervention strategies in the school setting on anthropometric and quality of diet outcomes by comparing and ranking outcomes in a network meta-analysis.Design Systematic review and network meta-analysis.Data sources PubMed, Cochrane Central Register of Controlled Trials (CENTRAL), Web of Science, Education Resources Information Centre (ERIC), PsycInfo, CAB Abstracts, Campbell Library, Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) BiblioMap, Australian Education Index, Joanna Briggs Institute Evidence-Based Practice (JBI EBP) database, Practice-based Evidence in Nutrition (PEN) database, ClinicalTrials.gov, Current Controlled Trials, and World Health Organization International Clinical Trials Registry Platform.Eligibility criteria for selecting studies A systematic literature search was performed from inception to 2 May 2022. Cluster randomised controlled trials meeting these study criteria were included: generally healthy school students aged 4-18 years; intervention with ≥1 nutritional components in a school setting; and studies that assessed anthropometric measures (eg, body mass index, body fat) or measures related to the quality of diet (eg, intake of fruit and vegetables), or both. Random effects pairwise meta-analyses and network meta-analyses were performed with a frequentist approach. P scores, a frequentist analogue to surface under the cumulative ranking curve, ranging from 0 to 1 (indicating worst and best ranked interventions, respectively) were calculated. Risk of bias was assessed with Cochrane’s RoB 2 tool. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework was used to rate the certainty of evidence.Results 51 cluster randomised controlled trials involving 75 954 participants and seven intervention nodes were included. Inconsistency could not be assessed (except for intake of fruit and vegetables) because the network meta-analyses were based mainly on star shaped networks with no direct evidence for specific pairs of nutritional interventions. Overall, little or no evidence was found to support a difference in body mass index, body weight, body fat, or waist circumference and moderate improvements in intake of fruit and vegetables with nutritional interventions in a school setting. Low to moderate certainty of evidence further suggested that multicomponent nutritional interventions likely reduced the prevalence (odds ratio 0.66, 95% confidence interval 0.55 to 0.80) and incidence (0.67, 0.47 to 0.96) of overweight compared with a control group. Based on low certainty of evidence, nutrition education and multicomponent interventions may be more effective than a control group (ie, usual practice) for increasing intake of fruit and vegetables. Multicomponent nutritional interventions were ranked the most effective for reducing body mass index (P score 0.76) and intake of fat (0.82). Nutrition education was ranked as best for body mass index z score (0.99), intake of fruit and vegetables (0.82), intake of fruit (0.92), and intake of vegetables (0.88).Conclusions The findings suggest that nutritional interventions in school settings may improve anthropometric and quality of diet measures, potentially contributing to the prevention of overweight and obesity in childhood and adolescence. The findings should be interpreted with caution because the certainty of evidence was often rated as low. The results of the network meta-analysis could be used by policy makers in developing and implementing effective, evidence based nutritional intervention strategies in the school setting.Systematic review registration PROSPERO CRD42020220451.

Objective The Collaboration for Evidence-Based Healthcare and Public Health in sub-Saharan Africa (CEBHA+), a research network, aims to build capacities for evidence-based healthcare. Hypertension (HTN) and diabetes mellitus (DM) are two priority areas of the network, both are major causes of burden of disease in this region. This review aimed to: (1) identify existing evidence-based guidelines for HTN and DM, (2) map their recommendations and (3) assess their quality.Setting Sub-Saharan Africa.Design Scoping review.Methods Systematic searches for evidence-based guidelines, developed with systematic review of evidence and certainty of evidence assessment, were undertaken in electronic databases and grey literature, and ministries of health of all countries in this region were contacted. Included guidelines were assessed with the Appraisal of Guidelines for research and evaluation II (AGREE-II) tool. Searches were conducted between 7 December 2021 and 14 January 2022. Results are presented descriptively.Results 66 potentially relevant guidelines were identified, developed in 23, out of 49 sub-Saharan African countries. Of these, only two guidelines (on DM) reported the use of systematic review of evidence and certainty of evidence assessment. Their quality appraisal showed that both have relatively similar scores on domains of AGREE-II, with higher scores on Scope and Purpose and Clarity and Presentation domains, and lower on Stakeholder Involvement, Applicability, Rigour of Development and Editorial independence domains. The overall scores of both guidelines were 50% and 58%, respectively.Conclusions Less than half of the countries in sub-Saharan Africa developed and published their own guidelines for HTN or DM. The quality appraisal showed that the two included guidelines scored relatively low in several crucial domains of AGREE-II. Countries in this region could consider adopting or adapting already published high-quality recommendations, in order to facilitate a more efficient and faster development of much needed trustworthy evidence-based guidance.

Abstract Background Although quality indicators are frequently derived from guidelines, there is a substantial gap in collaboration between the corresponding parties. To optimise workflow, guideline recommendations and quality assurance should be aligned methodologically and practically. Learning from the European Commission Initiative on Breast Cancer (ECIBC), our objective was to bring the key knowledge and most important considerations from both worlds together to inform European Commission future initiatives. Methods We undertook several steps to address the problem. First, we conducted a feasibility study that included a survey, interviews and a review of manuals for an integrated guideline and quality assurance (QA) scheme that would support the European Commission. The feasibility study drew from an assessment of the ECIBC experience that followed commonly applied strategies leading to separation of the guideline and QA development processes. Secondly, we used results of a systematic review to inform our understanding of methodologies for integrating guideline and QA development. We then, in a third step, used the findings to prepare an evidence brief and identify key aspects of a methodological framework for integrating guidelines QA through meetings with key informants. Results Seven key themes emerged to be taken into account for integrating guidelines and QA schemes: (1) evidence-based integrated guideline and QA frameworks are possible, (2) transparency is key in clearly documenting the source and rationale for quality indicators, (3) intellectual and financial interests should be declared and managed appropriately, (4) selection processes and criteria for quality indicators need further refinement, (5) clear guidance on retirement of quality indicators should be included, (6) risks of an integrated guideline and QA Group can be mitigated, and (7) an extension of the GIN-McMaster Guideline Development Checklist should incorporate QA considerations. Discussion We concluded that the work of guideline and QA developers can be integrated under a common methodological framework and we provided key findings and recommendations. These two worlds, that are fundamental to improving health, can both benefit from integration.

Objective We investigated characteristics of systematic reviews (SRs) assessing measures to prevent COVID-19 by (1) identifying SR registrations in Prospective Register of Systematic Reviews (PROSPERO), (2) identifying published SRs in COVID-19 Living Overview of the Evidence (L-OVE) and (3) estimating the proportion of PROSPERO registrations published as full SR between 8 and 16 months after registration.Study design This meta-research study is part of the German CEOsys project, www.covid-evidenz.de. We searched PROSPERO entries registered between 1 January 2020 and 31 August 2020, and we searched COVID-19 L-OVE for published SRs (search date: 5 May 2021) focusing on measures to prevent COVID-19 and SARS-CoV-2 transmission. The two samples were screened for eligibility and key characteristics were extracted and summarised.Results Of 612 PROSPERO registrations, 47 focused on prevention and were included. The preventive measures included public health interventions (20), followed by personal protective equipment (10), vaccinations (9) and others (8). In total, 13 of 47 (28%) PROSPERO registrations had been published as full SR (as preprint only (6), as peer-reviewed article only (6), as preprint and peer-reviewed article (1)). Median time between PROSPERO registration and publication was 5 months for peer-reviewed SRs and 2 months for preprints.Of the 2182 entries identified in COVID-19 L-OVE, 51 published SRs focused on prevention and were included. Similar to the PROSPERO sample, most published SRs focused on public health interventions (21). The number of included primary studies ranged between 0 and 64 (median: 7). Nine published SRs did not include any studies because of a lack of primary studies.Conclusion Considering the urgent information needs of policymakers and the public, our findings reveal the high-speed publication of preprints and lack of primary studies in the beginning of the COVID-19 crisis. Further meta-research on COVID-19 SRs is important to improve research efficiency among researchers across the world.PROSPERO registration number CRD42021240423.

Objectives Comprehensive protocols are key for the planning and conduct of randomised clinical trials (RCTs). Evidence of low reporting quality of RCT protocols led to the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist in 2013. We aimed to examine the quality of reporting of RCT protocols from three countries before and after the publication of the SPIRIT checklist.Design Repeated cross sectional study.Setting Swiss, German and Canadian research ethics committees (RECs).Participants RCT protocols approved by RECs in 2012 (n=257) and 2016 (n=292).Primary and secondary outcome measures The primary outcomes were the proportion of reported SPIRIT items per protocol and the proportion of trial protocols reporting individual SPIRIT items. We compared these outcomes in protocols approved in 2012 and 2016, and built regression models to explore factors associated with adherence to SPIRIT. For each protocol, we also extracted information on general trial characteristics and assessed whether individual SPIRIT items were reportedResults The median proportion of reported SPIRIT items among RCT protocols showed a non-significant increase from 72% (IQR, 63%–79%) in 2012 to 77% (IQR, 68%–82%) in 2016. However, in a preplanned subgroup analysis, we detected a significant improvement in investigator-sponsored protocols: the median proportion increased from 64% (IQR, 55%–72%) in 2012 to 76% (IQR, 64%–83%) in 2016, while for industry-sponsored protocols median adherence was 77% (IQR 72%–80%) for both years. The following trial characteristics were independently associated with lower adherence to SPIRIT: single-centre trial, no support from a clinical trials unit or contract research organisation, and investigator-sponsorship.Conclusions In 2012, industry-sponsored RCT protocols were reported more comprehensively than investigator-sponsored protocols. After publication of the SPIRIT checklist, investigator-sponsored protocols improved to the level of industry-sponsored protocols, which did not improve.

BackgroundWe previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs.Methods and findingsWe included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations.ConclusionsWe have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research.

BackgroundPost-entry studies are a key element in managed entry agreements and aim at generating evidence about the additional benefit of new medical interventions before reimbursement decisions are made. This study evaluates the willingness of different stakeholder groups to engage post-entry in studies for benefit assessment and to assess differences in their willingness by study type, i.e. randomised controlled trial or observational study.MethodsWe conducted a cross-sectional, web-based survey with a self-administrated questionnaire in German language. We disseminated invitations to patients, patient representatives, healthcare providers, trialists & scientists and representatives of the medical private sector, using a snowball system, public contact details of German associations and organisations, and social media. We analysed quantitative data descriptively and qualitative data inductively.ResultsData of 154 respondents were available for analysis. The majority (>85%) was willing to engage in the studies in general, and regarding different study types. Scientists reported a higher willingness to conduct and support RCTs (p = 0.01) as compared to observational studies. Representatives of the private sector were mainly willing to support, but not to carry out post-entry studies. Stakeholders frequently mentioned that potential personal benefit and altruistic motives were relevant for their decision to engage in studies. Practical inconveniences, poor integration into daily life, high demand for time and personnel, and lack of resources were commonly mentioned barriers.Discussion and conclusionStakeholders clearly reported to be willing to engage in post-entry studies for benefit assessment. Self-reported willingness to participate in and support for studies seems higher than practical recruitment rates. The survey might be subject to survey error and self-enhancement of participants. Inquiring about the willingness of hypothetical studies might have caused participants to report higher willingness. Motives for and against participation may be possible starting points for approaches to overcome recruitment difficulties and facilitate successful study conduct.

Objective To explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients.Study design and setting Using individual participant data from four cluster-randomised trials conducted in the Netherlands and Germany, we used logistic regression to develop a prognostic model to predict all-cause HAs within a 6-month follow-up period. A stratified intercept was used to account for heterogeneity in baseline risk between the studies. The model was validated both internally and by using internal-external cross-validation (IECV).Results Prior HAs, physical components of the health-related quality of life comorbidity index, and medication-related variables were used in the final model. While achieving moderate discriminatory performance, internal bootstrap validation revealed a pronounced risk of overfitting. The results of the IECV, in which calibration was highly variable even after accounting for between-study heterogeneity, agreed with this finding. Heterogeneity was equally reflected in differing baseline risk, predictor effects and absolute risk predictions.Conclusions Predictor effect heterogeneity and differing baseline risk can explain the limited external performance of HA prediction models. With such drivers known, model adjustments in external validation settings (eg, intercept recalibration, complete updating) can be applied more purposefully.Trial registration number PROSPERO id: CRD42018088129.

Today, keeping up with the fast evolving evidence is more challenging than ever for practising physicians. A huge number of studies are published every day, and it is no longer possible to read all the relevant individual studies. Many physicians prefer attending continuing medical education (CME) to reading international scientific publications. Consequently, it is critical that CME is based on the best available evidence and presented in an unbiased manner free of conflicts of interest. Systematic reviews and Cochrane reviews in particular can thus provide a valuable resource of up-to-date and high-quality information on health care questions for CME providers. Of note, systematic reviews might become outdated quickly. Furthermore, some systematic reviews are fraught with limitations such as poor methodology and conduct or incomplete and misleading reporting. This article provides a brief overview of systematic reviews and Cochrane reviews, outlines how systematic reviews can be “kept alive” using today’s digital opportunities and points to several common problems of systematic reviews with suggestions for solutions.

Introduction The US opioid crisis and increasing prescription rates in Europe suggest inappropriate risk perceptions and behaviours of people who prescribe, take or advise on opioids: physicians, patients and pharmacists. Findings from cognitive and decision science in areas other than drug safety suggest that people’s risk perception and behaviour can differ depending on whether they learnt about a risk through personal experience or description. Experiencing the risk of overutilising opioids among patients with chronic non-cancer pain in ambulatory care (ERONA) is the first-ever conducted trial that aims at investigating the effects of these two modes of learning on individuals’ risk perception and behaviour in the long-term administration of WHO-III opioids in chronic non-cancer pain.Methods and analysis ERONA—an exploratory, randomised controlled online survey intervention trial with two parallel arms—will examine the opioid-associated risk perception and behaviour of four groups involved in the long-term administration of WHO-III opioids: (1) family physicians, (2) physicians specialised in pain therapy, (3) patients with chronic (≥3 months) non-cancer pain and (4) pharmacists who regularly dispense narcotic substances. Participants will be randomly assigned to one of two online risk education interventions, description based or experiencebased. Both interventions will present the best medical evidence available. Participants will be queried at baseline and after intervention on their risk perception of opioids’ benefit–harm ratio, their medical risk literacy and their current/intended risk behaviour (in terms of prescribing, taking or counselling, depending on study group). A follow-up will occur after 9 months, when participants will be queried on their actual risk behaviour. The study was developed by the authors and will be conducted by the market research institution IPSOS Health.Ethics and dissemination The study was approved by the Institutional Review Board of the Max Planck Institute for Human Development. Results will be disseminated through peer-reviewed journals, conference presentations and social media.Trial registration number DRKS00020358.

Abstract Background In low- and middle-income countries (LMICs), the burden of non-communicable diseases (NCDs) is growing against an existing burden of other diseases such as HIV/AIDS. Integrated models of care can help address the rising burden of multi-morbidity. Although integration of care can occur at various levels and has been defined in numerous ways, our aim is to assess the effects of integration of service delivery at primary healthcare level in LMICs. Methods We will consider randomised controlled trials (RCTs), cluster RCTs, non-randomised trials, controlled before-after studies and interrupted time series that examine integrated models of care among people with multi-morbidities, of which diabetes or hypertension is one, living in LMICs. We will compare fully integrated models of care to stand-alone care, partially integrated models of care to stand-alone care and fully integrated models to partially integrated models of care. Primary outcomes include all-cause mortality, disease-specific morbidity, HbA1c, systolic blood pressure and cholesterol levels. Secondary outcomes include access to care, retention in care, adherence, continuity of care, quality of care and cost of care. We will conduct a comprehensive search in the following databases: MEDLINE, EMBASE, the Cochrane Central Register of Control Trials, LILACS, Africa-Wide Information, CINAHL and Web of Science. In addition, we will search trial registries, relevant conference abstracts and check references lists of included studies. Selection of studies, data extraction and assessment of risk of bias will be performed independently by two review authors. We will resolve discrepancies through discussion with a third author. We will contact study authors in case of missing data. If included studies are sufficiently homogenous, we will pool results in a meta-analysis. Clinical heterogeneity related to the population, intervention, outcomes and context will be documented in table format and explored through subgroup analysis. We will assess χ 2 and I 2 tests for statistical heterogeneity. We will use GRADE to make judgements about the certainty of evidence and present findings in a summary of findings table. Discussion In light of limited evidence on the provision of comprehensive care for diabetes and hypertension, and its comorbidity in LMCIs, we believe that the findings of this systematic review will provide a synthesis of evidence on effective models of integrated care for diabetes and hypertension and their comorbidities at primary healthcare level. This will enable policy-makers to device policies and programs that are evidence informed. Systematic review registration PROSPERO CRD42018099314.

BACKGROUND:The debate about the value of biomedical publications led to recommendations for improving reporting quality. It is unclear to what extent these recommendations have been endorsed by journals. We analyzed whether specific recommendations were included in author instructions, which journal characteristics were associated with their endorsement, how endorsement of the domains changed and whether endorsement was associated with change of impact factor between 2010 and 2015. METHODS:We considered two study samples consisting of "Hematology" and "Oncology" journals of the Journal Citation Report 2008 and 2014, respectively. We extracted information regarding endorsement of the (1) recommendations of the International Committee of Medical Journal Editors, of (2) reporting guidelines, (3) requirement for trial registration and (4) disclosure of conflicts of interest. Data extraction was done by reading the author instructions before conducting a text search with keywords. We calculated a global generalized linear mixed effects model for endorsement of each of the four domains followed by separate multivariable logistic regression models and a longitudinal analysis. We defined endorsement as the author instructions saying that they approve the use of the recommendations. RESULTS:In 2015, the ICMJE recommendations were mentioned in author instructions of 156 journals (67.5%). CONSORT was referred to by 77 journals (33.3%); MOOSE, PRISMA, STARD and STROBE were referred to by less than 15% of journals. There were 99 journals (42.9%) that recommended or required trial registration, 211 (91.3%) required authors to disclose conflicts of interest. Journal impact factor, journal start year and geographical region were positively associated with endorsement of any of the four domains. The overall endorsement of all domains increased between 2010 and 2015. The endorsement of any domain in 2010 seemed to be associated with an increased impact factor in 2014. CONCLUSION:Hematology and oncology journals endorse major recommendations to various degrees. Endorsement is increasing slowly over time and might be positively associated with the journals' impact factor.

BACKGROUND:A meta-analysis as part of a systematic review aims to provide a thorough, comprehensive and unbiased statistical summary of data from the literature. However, relevant study results could be missing from a meta-analysis because of selective publication and inadequate dissemination. If missing outcome data differ systematically from published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention effect. As part of the EU-funded OPEN project (www.open-project.eu) we conducted a systematic review that assessed whether the inclusion of data that were not published at all and/or published only in the grey literature influences pooled effect estimates in meta-analyses and leads to different interpretation. METHODS AND FINDINGS:Systematic review of published literature (methodological research projects). Four bibliographic databases were searched up to February 2016 without restriction of publication year or language. Methodological research projects were considered eligible for inclusion if they reviewed a cohort of meta-analyses which (i) compared pooled effect estimates of meta-analyses of health care interventions according to publication status of data or (ii) examined whether the inclusion of unpublished or grey literature data impacts the result of a meta-analysis. Seven methodological research projects including 187 meta-analyses comparing pooled treatment effect estimates according to different publication status were identified. Two research projects showed that published data showed larger pooled treatment effects in favour of the intervention than unpublished or grey literature data (Ratio of ORs 1.15, 95% CI 1.04-1.28 and 1.34, 95% CI 1.09-1.66). In the remaining research projects pooled effect estimates and/or overall findings were not significantly changed by the inclusion of unpublished and/or grey literature data. The precision of the pooled estimate was increased with narrower 95% confidence interval. CONCLUSIONS:Although we may anticipate that systematic reviews and meta-analyses not including unpublished or grey literature study results are likely to overestimate the treatment effects, current empirical research shows that this is only the case in a minority of reviews. Therefore, currently, a meta-analyst should particularly consider time, effort and costs when adding such data to their analysis. Future research is needed to identify which reviews may benefit most from including unpublished or grey data.

Abstract Following publication of the original article [1], the author reported that their family name was misspelled.

BackgroundLittle is known about publication agreements between industry and academic investigators in trial protocols and the consistency of these agreements with corresponding statements in publications. We aimed to investigate (i) the existence and types of publication agreements in trial protocols, (ii) the completeness and consistency of the reporting of these agreements in subsequent publications, and (iii) the frequency of co-authorship by industry employees.Methods and findingsWe used a retrospective cohort of randomized clinical trials (RCTs) based on archived protocols approved by six research ethics committees between 13 January 2000 and 25 November 2003. Only RCTs with industry involvement were eligible. We investigated the documentation of publication agreements in RCT protocols and statements in corresponding journal publications. Of 647 eligible RCT protocols, 456 (70.5%) mentioned an agreement regarding publication of results. Of these 456, 393 (86.2%) documented an industry partner's right to disapprove or at least review proposed manuscripts; 39 (8.6%) agreements were without constraints of publication. The remaining 24 (5.3%) protocols referred to separate agreement documents not accessible to us. Of those 432 protocols with an accessible publication agreement, 268 (62.0%) trials were published. Most agreements documented in the protocol were not reported in the subsequent publication (197/268 [73.5%]). Of 71 agreements reported in publications, 52 (73.2%) were concordant with those documented in the protocol. In 14 of 37 (37.8%) publications in which statements suggested unrestricted publication rights, at least one co-author was an industry employee. In 25 protocol-publication pairs, author statements in publications suggested no constraints, but 18 corresponding protocols documented restricting agreements.ConclusionsPublication agreements constraining academic authors' independence are common. Journal articles seldom report on publication agreements, and, if they do, statements can be discrepant with the trial protocol.

Qualitative research findings are increasingly used to inform decision-making. Research has indicated that not all quantitative research on the effects of interventions is disseminated or published. The extent to which qualitative researchers also systematically underreport or fail to publish certain types of research findings, and the impact this may have, has received little attention.A survey was delivered online to gather data regarding non-dissemination and dissemination bias in qualitative research. We invited relevant stakeholders through our professional networks, authors of qualitative research identified through a systematic literature search, and further via snowball sampling.1032 people took part in the survey of whom 859 participants identified as researchers, 133 as editors and 682 as peer reviewers. 68.1% of the researchers said that they had conducted at least one qualitative study that they had not published in a peer-reviewed journal. The main reasons for non-dissemination were that a publication was still intended (35.7%), resource constraints (35.4%), and that the authors gave up after the paper was rejected by one or more journals (32.5%). A majority of the editors and peer reviewers "(strongly) agreed" that the main reasons for rejecting a manuscript of a qualitative study were inadequate study quality (59.5%; 68.5%) and inadequate reporting quality (59.1%; 57.5%). Of 800 respondents, 83.1% "(strongly) agreed" that non-dissemination and possible resulting dissemination bias might undermine the willingness of funders to support qualitative research. 72.6% and 71.2%, respectively, "(strongly) agreed" that non-dissemination might lead to inappropriate health policy and health care.The proportion of non-dissemination in qualitative research is substantial. Researchers, editors and peer reviewers play an important role in this. Non-dissemination and resulting dissemination bias may impact on health care research, practice and policy. More detailed investigations on patterns and causes of the non-dissemination of qualitative research are needed.

BackgroundSystematic reviews of preclinical studies, in vivo animal experiments in particular, can influence clinical research and thus even clinical care. Dissemination bias, selective dissemination of positive or significant results, is one of the major threats to validity in systematic reviews also in the realm of animal studies. We conducted a systematic review to determine the number of published systematic reviews of animal studies until present, to investigate their methodological features especially with respect to assessment of dissemination bias, and to investigate the citation of preclinical systematic reviews on clinical research.MethodsEligible studies for this systematic review constitute systematic reviews that summarize in vivo animal experiments whose results could be interpreted as applicable to clinical care. We systematically searched Ovid Medline, Embase, ToxNet, and ScienceDirect from 1st January 2009 to 9th January 2013 for eligible systematic reviews without language restrictions. Furthermore we included articles from two previous systematic reviews by Peters et al. and Korevaar et al.ResultsThe literature search and screening process resulted in 512 included full text articles. We found an increasing number of published preclinical systematic reviews over time. The methodological quality of preclinical systematic reviews was low. The majority of preclinical systematic reviews did not assess methodological quality of the included studies (71%), nor did they assess heterogeneity (81%) or dissemination bias (87%). Statistics quantifying the importance of clinical research citing systematic reviews of animal studies showed that clinical studies referred to the preclinical research mainly to justify their study or a future study (76%).DiscussionPreclinical systematic reviews may have an influence on clinical research but their methodological quality frequently remains low. Therefore, systematic reviews of animal research should be critically appraised before translating them to a clinical context.

BackgroundMany clinical studies are ultimately not fully published in peer-reviewed journals. Underreporting of clinical research is wasteful and can result in biased estimates of treatment effect or harm, leading to recommendations that are inappropriate or even dangerous.MethodsWe assembled a cohort of clinical studies approved 2000-2002 by the Research Ethics Committee of the University of Freiburg, Germany. Published full articles were searched in electronic databases and investigators contacted. Data on study characteristics were extracted from protocols and corresponding publications. We characterized the cohort, quantified its publication outcome and compared protocols and publications for selected aspects.ResultsOf 917 approved studies, 807 were started and 110 were not, either locally or as a whole. Of the started studies, 576 (71%) were completed according to protocol, 128 (16%) discontinued and 42 (5%) are still ongoing; for 61 (8%) there was no information about their course. We identified 782 full publications corresponding to 419 of the 807 initiated studies; the publication proportion was 52% (95% CI: 0.48-0.55). Study design was not significantly associated with subsequent publication. Multicentre status, international collaboration, large sample size and commercial or non-commercial funding were positively associated with subsequent publication. Commercial funding was mentioned in 203 (48%) protocols and in 205 (49%) of the publications. In most published studies (339; 81%) this information corresponded between protocol and publication. Most studies were published in English (367; 88%); some in German (25; 6%) or both languages (27; 6%). The local investigators were listed as (co-)authors in the publications corresponding to 259 (62%) studies.ConclusionHalf of the clinical research conducted at a large German university medical centre remains unpublished; future research is built on an incomplete database. Research resources are likely wasted as neither health care professionals nor patients nor policy makers can use the results when making decisions.

BACKGROUND: The synthesis of published research in systematic reviews is essential when providing evidence to inform clinical and health policy decision-making. However, the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted (dissemination bias). To investigate the extent of dissemination bias we conducted a systematic review that determined the proportion of studies published as peer-reviewed journal articles and investigated factors associated with full publication in cohorts of studies (i) approved by research ethics committees (RECs) or (ii) included in trial registries. METHODS AND FINDINGS: Four bibliographic databases were searched for methodological research projects (MRPs) without limitations for publication year, language or study location. The searches were supplemented by handsearching the references of included MRPs. We estimated the proportion of studies published using prediction intervals (PI) and a random effects meta-analysis. Pooled odds ratios (OR) were used to express associations between study characteristics and journal publication. Seventeen MRPs (23 publications) evaluated cohorts of studies approved by RECs; the proportion of published studies had a PI between 22% and 72% and the weighted pooled proportion when combining estimates would be 46.2% (95% CI 40.2%-52.4%, I2 = 94.4%). Twenty-two MRPs (22 publications) evaluated cohorts of studies included in trial registries; the PI of the proportion published ranged from 13% to 90% and the weighted pooled proportion would be 54.2% (95% CI 42.0%-65.9%, I2 = 98.9%). REC-approved studies with statistically significant results (compared with those without statistically significant results) were more likely to be published (pooled OR 2.8; 95% CI 2.2-3.5). Phase-III trials were also more likely to be published than phase II trials (pooled OR 2.0; 95% CI 1.6-2.5). The probability of publication within two years after study completion ranged from 7% to 30%. CONCLUSIONS: A substantial part of the studies approved by RECs or included in trial registries remains unpublished. Due to the large heterogeneity a prediction of the publication probability for a future study is very uncertain. Non-publication of research is not a random process, e.g., it is associated with the direction of study findings. Our findings suggest that the dissemination of research findings is biased.

IntroductionReporting guidelines (e.g. CONSORT) have been developed as tools to improve quality and reduce bias in reporting research findings. Trial registration has been recommended for countering selective publication. The International Committee of Medical Journal Editors (ICMJE) encourages the implementation of reporting guidelines and trial registration as uniform requirements (URM). For the last two decades, however, biased reporting and insufficient registration of clinical trials has been identified in several literature reviews and other investigations. No study has so far investigated the extent to which author instructions in psychiatry journals encourage following reporting guidelines and trial registration.MethodPsychiatry Journals were identified from the 2011 Journal Citation Report. Information given in the author instructions and during the submission procedure of all journals was assessed on whether major reporting guidelines, trial registration and the ICMJE's URM in general were mentioned and adherence recommended.ResultsWe included 123 psychiatry journals (English and German language) in our analysis. A minority recommend or require 1) following the URM (21%), 2) adherence to reporting guidelines such as CONSORT, PRISMA, STROBE (23%, 7%, 4%), or 3) registration of clinical trials (34%). The subsample of the top-10 psychiatry journals (ranked by impact factor) provided much better but still improvable rates. For example, 70% of the top-10 psychiatry journals do not ask for the specific trial registration number.DiscussionUnder the assumption that better reported and better registered clinical research that does not lack substantial information will improve the understanding, credibility, and unbiased translation of clinical research findings, several stakeholders including readers (physicians, patients), authors, reviewers, and editors might benefit from improved author instructions in psychiatry journals. A first step of improvement would consist in requiring adherence to the broadly accepted reporting guidelines and to trial registration.

BACKGROUND: Synthesizing research evidence using systematic and rigorous methods has become a key feature of evidence-based medicine and knowledge translation. Systematic reviews (SRs) may or may not include a meta-analysis depending on the suitability of available data. They are often being criticised as 'secondary research' and denied the status of original research. Scientific journals play an important role in the publication process. How they appraise a given type of research influences the status of that research in the scientific community. We investigated the attitudes of editors of core clinical journals towards SRs and their value for publication. METHODS: We identified the 118 journals labelled as "core clinical journals" by the National Library of Medicine, USA in April 2009. The journals' editors were surveyed by email in 2009 and asked whether they considered SRs as original research projects; whether they published SRs; and for which section of the journal they would consider a SR manuscript. RESULTS: The editors of 65 journals (55%) responded. Most respondents considered SRs to be original research (71%) and almost all journals (93%) published SRs. Several editors regarded the use of Cochrane methodology or a meta-analysis as quality criteria; for some respondents these criteria were premises for the consideration of SRs as original research. Journals placed SRs in various sections such as "Review" or "Feature article". Characterization of non-responding journals showed that about two thirds do publish systematic reviews. DISCUSSION: Currently, the editors of most core clinical journals consider SRs original research. Our findings are limited by a non-responder rate of 45%. Individual comments suggest that this is a grey area and attitudes differ widely. A debate about the definition of 'original research' in the context of SRs is warranted.

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited Conclusions: The size and speed of the COVID-19 clinical trials agenda is unprecedented. However, most trials were small investigating a small fraction of treatment options. The feasibility of this research agenda is questionable, and many trials may end in futility, wasting research resources. Much better coordination is needed to respond to global health threats.

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited Conclusions: The size and speed of the COVID-19 clinical trials agenda is unprecedented. However, most trials were small investigating a small fraction of treatment options. The feasibility of this research agenda is questionable, and many trials may end in futility, wasting research resources. Much better coordination is needed to respond to global health threats.