Search

Background: Trial registration aims to address potential bias from selective or non-reporting of findings, and therefore has a vital role in promoting transparency and accountability of clinical research. In this study, we aim to investigate the influence of trial registration on estimated harm effects in randomized controlled trials of medication interventions. Methods: We searched PubMed for systematic reviews and meta-analyses of randomized trials on medication harms indexed between January 1, 2015, and January 1, 2020. To be included in the analyses, eligible meta-analyses should have at least five randomized trials with distinct registration statuses (i.e., prospectively registered, retrospectively registered, and non-registered) and 2 by 2 table data for adverse events for each trial. To control for potential confounding, trials in each meta-analysis were analyzed within confounder-harmonized groups (e.g., dosage) identified using the Directed Acyclic Graph method. The harm estimates arising from the trials with different registration statuses were compared within the confounder-harmonized groups using hierarchical linear regression. Results are shown as ratio of odds ratio (OR) and 95% confidence interval (CI). Results: The dataset consists of 629 meta-analyses of harms with 10,069 trials. Of these trials, 74.3% were registered, and 23.9% were not registered, and for those registered, 70.6% were prospectively registered, while 26.3% were retrospectively registered. In comparison to prospectively registered trials, both non-registered trials (ratio of OR = 0.82, 95%CI 0.68 to 0.98, P = 0.03) and retrospectively registered trials (ratio of OR = 0.75, 95%CI 0.66 to 0.86, P < 0.01) had lower OR for harms based on 69 and 126 confounders-harmonized groups. The OR of harms did not differ between retrospectively registered and non-registered trials (ratio of OR = 1.02, 95%CI 0.85 to 1.23, P = 0.83) based on 76 confounders-harmonized groups. Conclusions: Medication-related harms may be understated in non-registered trials, and there was no obvious evidence that retrospective registration had a demonstrable benefit in reducing such selective or absent reporting. Prospective registration is highly recommended for future trials. [ABSTRACT FROM AUTHOR]

We conducted a multi-pronged investigation of different types of reporting bias in autism early childhood intervention research. First, we investigated the prevalence of reporting failures of completed trials registered on clinicaltrials.gov, and found that only 7% of registered trials were updated with results on the registration platform and only 64% had associated published reports. Next, we investigated the extent to which inadequate reporting prevents inclusion in meta-analytic summary estimates by identifying reports of studies that were eligible for inclusion in a prior meta-analysis, and found that 25% were excluded due to inadequate reporting. Finally, we investigated selective reporting practices by analyzing the protocols of the studies included in the meta-analysis which had been registered on any trial registry and coding their timing, completeness, and consistency. We found that 23% of studies were pre-registered, 71% were late-registered, and 5% were registered at an unclear date. Only 8% of registrations specified all of the necessary components. Evidence of selective reporting was common; 36% failed to report a registered outcome, 61% reported unregistered outcomes, 23% switched primary and secondary outcomes, and 43% had assessment timepoints that differed from registration specification. Given the inadequacy of registration and reporting practices, we offer practical recommendations to facilitate improvement for the field of autism research. When researchers fail to report their findings or only report some of their findings, it can make it difficult for clinicians to provide effective intervention recommendations. However, no one has examined whether this is a problem in studies of early childhood autism interventions. We studied how researchers that study early childhood autism interventions report their findings. We found that most researchers did not register their studies when they were supposed to (before the start of the study), and that many researchers did not provide all of the needed information in the registration. We also found that researchers frequently did not publish their findings when their studies were complete. When we looked at published reports, we found that many of the studies did not report enough information, and that many studies were reported differently from their registrations, suggesting that researchers were selectively reporting positive outcomes and ignoring or misrepresenting less positive outcomes. Because we found so much evidence that researchers are failing to report their findings quickly and correctly, we suggested some practical changes to make it better. [ABSTRACT FROM AUTHOR]

Abstract Background Trial registration aims to address potential bias from selective or non-reporting of findings, and therefore has a vital role in promoting transparency and accountability of clinical research. In this study, we aim to investigate the influence of trial registration on estimated harm effects in randomized controlled trials of medication interventions. Methods We searched PubMed for systematic reviews and meta-analyses of randomized trials on medication harms indexed between January 1, 2015, and January 1, 2020. To be included in the analyses, eligible meta-analyses should have at least five randomized trials with distinct registration statuses (i.e., prospectively registered, retrospectively registered, and non-registered) and 2 by 2 table data for adverse events for each trial. To control for potential confounding, trials in each meta-analysis were analyzed within confounder-harmonized groups (e.g., dosage) identified using the Directed Acyclic Graph method. The harm estimates arising from the trials with different registration statuses were compared within the confounder-harmonized groups using hierarchical linear regression. Results are shown as ratio of odds ratio (OR) and 95% confidence interval (CI). Results The dataset consists of 629 meta-analyses of harms with 10,069 trials. Of these trials, 74.3% were registered, and 23.9% were not registered, and for those registered, 70.6% were prospectively registered, while 26.3% were retrospectively registered. In comparison to prospectively registered trials, both non-registered trials (ratio of OR = 0.82, 95%CI 0.68 to 0.98, P = 0.03) and retrospectively registered trials (ratio of OR = 0.75, 95%CI 0.66 to 0.86, P

There is a need to create and publish trusted evidence that will inform clinical decisions. Librarians can play an important role in this process, as they can oversee the registration and reporting of primary and secondary studies, take an active part in data synthesis and contribute to increasing the quality, completeness and reliability of existing data.

Using a sample of 70,399 published p‐values from 192 meta‐analyses, we empirically estimate the counterfactual distribution of p‐values in the absence of any biases. Comparing observed p‐values with counterfactually expected p‐values allows us to estimate how many p‐values are published as being statistically significant when they should have been published as non‐significant. We estimate the extent of selectively reported p‐values to range between 57.7% and 71.9% of the significant p‐values. The counterfactual p‐value distribution also allows us to assess shifts of p‐values along the entire distribution of published p‐values, revealing that particularly very small p‐values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies. [ABSTRACT FROM AUTHOR]

Copyright of Revija Šport is the property of Sport: Revija Za Teoreticna in Praticna Vprasanja Sporta and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

As traditionally conceived, publication bias arises from selection operating on a collection of individually unbiased estimates. A canonical form of such selection across studies (SAS) is the preferential publication of affirmative studies (i.e., those with significant, positive estimates) versus nonaffirmative studies (i.e., those with nonsignificant or negative estimates). However, meta-analyses can also be compromised by selection within studies (SWS), in which investigators “p-hack” results within their study to obtain an affirmative estimate. Published estimates can then be biased even conditional on affirmative status, which comprises the performance of existing methods that only consider SAS. We propose two new analysis methods that accommodate joint SAS and SWS; both analyze only the published nonaffirmative estimates. First, we propose estimating the underlying meta-analytic mean by fitting “right-truncated meta-analysis” (RTMA) to the published nonaffirmative estimates. This method essentially imputes the entire underlying distribution of population effects. Second, we propose conducting a standard meta-analysis of only the nonaffirmative studies (MAN); this estimate is conservative (negatively biased) under weakened assumptions. We provide an R package (phacking) and website (metabias.io). Our proposed methods supplement existing methods by assessing the robustness of meta-analyses to joint SAS and SWS. [ABSTRACT FROM AUTHOR]

Background Systematic reviews that aim to synthesize evidence on the effects of interventions targeted at populations often include interrupted time-series (ITS) studies. However, the suppression of ITS studies or results within these studies (known as reporting bias) has the potential to bias conclusions drawn in such systematic reviews, with potential consequences for healthcare decision-making. Therefore, we aim to determine whether there is evidence of reporting bias among ITS studies. Methods We will conduct a search for published protocols of ITS studies and reports of their results in PubMed, MEDLINE, and Embase up to December 31, 2022. We contact the authors of the ITS studies to seek information about their study, including submission status, data for unpublished results, and reasons for non-publication or non-reporting of certain outcomes. We will examine if there is evidence of publication bias by examining whether time-to-publication is influenced by the statistical significance of the study’s results for the primary research question using Cox proportional hazards regression. We will examine whether there is evidence of discrepancies in outcomes by comparing those specified in the protocols with those in the reports of results, and we will examine whether the statistical significance of an outcome’s result is associated with how completely that result is reported using multivariable logistic regression. Finally, we will examine discrepancies between protocols and reports of results in the methods by examining the data collection processes, model characteristics, and statistical analysis methods. Discrepancies will be summarized using descriptive statistics. Discussion These findings will inform systematic reviewers and policymakers about the extent of reporting biases and may inform the development of mechanisms to reduce such biases.

Meta‐analyses can be compromised by studies' internal biases (e.g., confounding in nonrandomized studies) as well as publication bias. These biases often operate nonadditively: publication bias that favors significant, positive results selects indirectly for studies with more internal bias. We propose sensitivity analyses that address two questions: (1) "For a given severity of internal bias across studies and of publication bias, how much could the results change?"; and (2) "For a given severity of publication bias, how severe would internal bias have to be, hypothetically, to attenuate the results to the null or by a given amount?" These methods consider the average internal bias across studies, obviating specifying the bias in each study individually. The analyst can assume that internal bias affects all studies, or alternatively that it only affects a known subset (e.g., nonrandomized studies). The internal bias can be of unknown origin or, for certain types of bias in causal estimates, can be bounded analytically. The analyst can specify the severity of publication bias or, alternatively, consider a "worst‐case" form of publication bias. Robust estimation methods accommodate non‐normal effects, small meta‐analyses, and clustered estimates. As we illustrate by re‐analyzing published meta‐analyses, the methods can provide insights that are not captured by simply considering each bias in turn. An R package implementing the methods is available (multibiasmeta). [ABSTRACT FROM AUTHOR]

In their book ‘Nudge: Improving Decisions About Health, Wealth and Happiness’, Thaler & Sunstein (2009) argue that choice architectures are promising public policy interventions. This research programme motivated the creation of ‘nudge units’, government agencies which aim to apply insights from behavioural science to improve public policy. We closely examine a meta-analysis of the evidence gathered by two of the largest and most influential nudge units (DellaVigna & Linos (2022 Econometrica 90, 81–116 (doi:10.3982/ECTA18709))) and use statistical techniques to detect reporting biases. Our analysis shows evidence suggestive of selective reporting. We additionally evaluate the public pre-analysis plans from one of the two nudge units (Office of Evaluation Sciences). We identify several instances of excellent practice; however, we also find that the analysis plans and reporting often lack sufficient detail to evaluate (unintentional) reporting biases. We highlight several improvements that would enhance the effectiveness of the pre-analysis plans and reports as a means to combat reporting biases. Our findings and suggestions can further improve the evidence base for policy decisions.

Summary: Background: Selective reporting is a promising tool for antimicrobial stewardship, but in wound cultures, its effects on the use of antimicrobials are unknown. Our HUS Diagnostic Center Bacteriology laboratory refined its selective reporting protocol for wound cultures during 2017–2018. In this study we aimed to show our protocol's impact on the frequency of antimicrobial escalation. Methods: We performed a retrospective cohort study of patients in the wound-care ward of a primary-care hospital in Helsinki, Finland, from 2014 to 2016 (pre-intervention) and from 2019 to April 2021 (post-intervention). With the inclusion criterion being wound-culture collection, this provided us with 299 patients, of which 152 were in the pre-intervention group, and 147 were post-intervention. We collected the data from medical records and compared the pre-intervention- with the post-intervention group in terms of patient profiles, microbiology reports, antimicrobial treatment, and treatment outcomes. Findings: In the pre-intervention group 40% of the patients were male and 60% female and in the post-intervention group 49% and 51% respectively. The frequency of AST reported had decreased from 63% in the pre-intervention group to 37% post-intervention (OR 0.35, p

We conducted a systematic review of conference papers in social psychology at two large psychology conferences in Japan: the Japanese Psychological Association and the Japanese Society for Social Psychology. The conference papers were effectively not subjected to peer review; hence, they were suitable for testing if psychologists selectively reported statistically significant findings without pressure from journal editors and reviewers. We investigated the distributions of z-values converted from the p-values reported in the articles presented at the 2013 and 2018 conferences. The z-curve analyses suggest the existence of selective reporting by the authors in 2013. The expected discovery rate (EDR) was much lower than the observed discovery rate (ODR; 7% vs. 76%, respectively), and the 95% confidence interval (CI) did not include the ODR. However, this does not mean that the set of studies completely lacked evidential value. The expected replication rate (ERR) was 31%; this is significantly higher than 5%, which was expected under the null hypothesis of no effect. Changes were observed between 2013 and 2018. The ERR increased (31% to 44%), and the EDR almost doubled (7% to 13%). However, the estimation of the maximum false discovery rate (FDR; 68% in 2013 and 35% in 2018) suggested that a substantial proportion of the reported findings were false positives. Overall, while social psychologists in Japan engaged in selective reporting, this does not mean that the entire field was covered with false positives. In addition, slight signs of improvement were observed in how they reported their findings. Still, the evidential value of the target studies was weak, even in 2018, allowing for no optimism.

Background: Selective reporting has important value in antibiotic management. The purpose of this study was to explore the impact of AST selective reporting on prescribing behavior, so as to provide evidence for the implementation and improvement of selective reporting policies in microbiology laboratories at home and abroad. Methods: A cross-sectional study was conducted in a teaching tertiary hospital in China in July 2021. We designed selective reports and routine reports for urinary tract infections caused by Escherichia coli and lower respiratory tract infections caused by Pseudomonas aeruginosa. Questionnaires were conducted among participants by case vignettes, and 116 valid questionnaires were collected. The appropriateness rate of antibiotic prescription and the prescription rate of drugresistant antibiotics, cephalosporins, fluoroquinolones, and carbapenems were calculated and compared between the selective reporting group and the routine reporting group in each case. Results: In most cases, we found that AST selective reporting could increase the appropriateness rate of antibiotic prescription (p < 0.05) and reduce the drugresistant antibiotic prescription rate (p < 0.01), cephalosporin drug prescription rate (p < 0.05) and fluoroquinolone drug prescription rate (p < 0.01). Although the difference in carbapenems prescription rate was not significant, selective reporting could reduce the number of its prescriptions to some extent. Conclusion: AST selective reporting can help promote the appropriate use of antibiotics and reduce the use of broad-spectrum antibiotics. It is suggested to develop scientific and effective selective reporting practices and strengthen the two-way communication between clinicians and microbiology laboratories, thereby enabling microbiology laboratories to play a more important role in clinical antimicrobial management. [ABSTRACT FROM AUTHOR]

Abstract Collaborative efforts to directly replicate empirical studies in the medical and social sciences have revealed alarmingly low rates of replicability, a phenomenon dubbed the ‘replication crisis’. Poor replicability has spurred cultural changes targeted at improving reliability in these disciplines. Given the absence of equivalent replication projects in ecology and evolutionary biology, two inter-related indicators offer the opportunity to retrospectively assess replicability: publication bias and statistical power. This registered report assesses the prevalence and severity of small-study (i.e., smaller studies reporting larger effect sizes) and decline effects (i.e., effect sizes decreasing over time) across ecology and evolutionary biology using 87 meta-analyses comprising 4,250 primary studies and 17,638 effect sizes. Further, we estimate how publication bias might distort the estimation of effect sizes, statistical power, and errors in magnitude (Type M or exaggeration ratio) and sign (Type S). We show strong evidence for the pervasiveness of both small-study and decline effects in ecology and evolution. There was widespread prevalence of publication bias that resulted in meta-analytic means being over-estimated by (at least) 0.12 standard deviations. The prevalence of publication bias distorted confidence in meta-analytic results, with 66% of initially statistically significant meta-analytic means becoming non-significant after correcting for publication bias. Ecological and evolutionary studies consistently had low statistical power (15%) with a 4-fold exaggeration of effects on average (Type M error rates = 4.4). Notably, publication bias reduced power from 23% to 15% and increased type M error rates from 2.7 to 4.4 because it creates a non-random sample of effect size evidence. The sign errors of effect sizes (Type S error) increased from 5% to 8% because of publication bias. Our research provides clear evidence that many published ecological and evolutionary findings are inflated. Our results highlight the importance of designing high-power empirical studies (e.g., via collaborative team science), promoting and encouraging replication studies, testing and correcting for publication bias in meta-analyses, and adopting open and transparent research practices, such as (pre)registration, data- and code-sharing, and transparent reporting.

Copyright of Medical Library Forum is the property of Warszawski Uniwersytet Medyczny and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The use of statistical significance is under discussion. Many statisticians and researchers advocate for its retirement. Conversely, other statisticians and researchers think that its retirement would damage science. There is room for improvement in the use of hypothesis testing and p -values in biochemical sciences and omics. The selection of variables by statistical significance with solid cutoffs drives and may bias the biological interpretation of biochemical data. To obtain robust knowledge by comparing studies, it is essential to report thoroughly all results (both quantitative and categorical variables). Because of the big number of variables, the problems of selecting variables by statistical significance increase for omic studies. Over recent years many statisticians and researchers have highlighted that statistical inference would benefit from a better use and understanding of hypothesis testing, p -values, and statistical significance. We highlight three recommendations in the context of biochemical sciences. First recommendation: to improve the biological interpretation of biochemical data, do not use p -values (or similar test statistics) as thresholded values to select biomolecules. Second recommendation: to improve comparison among studies and to achieve robust knowledge, perform complete reporting of data. Third recommendation: statistical analyses should be reported completely with exact numbers (not as asterisks or inequalities). Owing to the high number of variables, a better use of statistics is of special importance in omic studies. [ABSTRACT FROM AUTHOR]

In the face of explosive growth of wireless data and electronic devices, cognitive radio (CR) technology is an effective solution to identify spectrum resources underutilized by the primary user (PU). Moreover, cooperative spectrum sensing (CSS) is the key function of CR technology to enhance the detection performance by exploiting spatial diversity via the observations of spatially located sensor nodes (SNs). However, in a classical cooperative paradigm, each SN is always required to report individual local decision about the PU status to the fusion center (FC) for the global decision-making. Such a static reporting model regardless of the surrounding environment is not conducive to achieving the cooperative gain and adapting to a large cognitive wireless sensor network (CWSN). In view of this, this paper proposes an efficient CSS to optimize the reporting time and the achievable throughput of CWSNs, in which consists of selective reporting, multi-bit quantization, data fusion, and dynamic parameter. First of all, SNs only report some necessary observations by means of selective reporting, and then encodes local observations into multi-bit binary codes and send them to the FC. Further, the received local observations are aggregated into a final decision about the PU status. To be specific, the bit number of binary data and the selective reporting range as a parameter are dynamically adjusted at each SN. At last, simulation results demonstrate that the proposed CSS algorithm significantly not only reduces reporting time, but also improves the achievable throughput of CWSNs while maintaining the CSS performance.

Abstract Background Studies reporting statistically significant effect estimates tend to be more frequently published compared to studies reporting non-significant or equivalent estimates. Consequently, this may lead to distortion of the literature. The aim of this study is to assess the prevalence of reporting statistically significant effect estimates in leading oral health journals and to explore associations between the effect estimates and record characteristics. Methods An electronic database search was undertaken of a selection of leading oral health journals including general oral health journals to identify primary oral health records published in 2019. Descriptive statistics and population average GEE logistic regression model was used to assess associations between articles reporting a statistically significant effect estimate and the record characteristics. Results In 1335 records, 82.4% records reported a statistically significant effect estimate. All speciality journals compared to general oral health journals were less likely to publish a record with significant effect estimates. Authors based in Asia or other (OR 1.49; 95% CI :1.02,2.19; p = 0.037) were more likely to report significant effect estimates compared to those based in Europe. Interventional (OR 0.35; 0.22,0.58; p

Collaborative efforts to directly replicate empirical studies in the medical and social sciences have revealed alarmingly low rates of replicability, a phenomenon dubbed the 'replication crisis'. Poor replicability has spurred cultural changes targeted at improving reliability in these disciplines. Given the absence of equivalent replication projects in ecology and evolutionary biology, two inter-related indicators offer the opportunity to retrospectively assess replicability: publication bias and statistical power. This registered report assesses the prevalence and severity of small-study (i.e., smaller studies reporting larger effect sizes) and decline effects (i.e., effect sizes decreasing over time) across ecology and evolutionary biology using 87 meta-analyses comprising 4,250 primary studies and 17,638 effect sizes. Further, we estimate how publication bias might distort the estimation of effect sizes, statistical power, and errors in magnitude (Type M or exaggeration ratio) and sign (Type S). We show strong evidence for the pervasiveness of both small-study and decline effects in ecology and evolution. There was widespread prevalence of publication bias that resulted in meta-analytic means being over-estimated by (at least) 0.12 standard deviations. The prevalence of publication bias distorted confidence in meta-analytic results, with 66% of initially statistically significant meta-analytic means becoming non-significant after correcting for publication bias. Ecological and evolutionary studies consistently had low statistical power (15%) with a 4-fold exaggeration of effects on average (Type M error rates = 4.4). Notably, publication bias reduced power from 23% to 15% and increased type M error rates from 2.7 to 4.4 because it creates a non-random sample of effect size evidence. The sign errors of effect sizes (Type S error) increased from 5% to 8% because of publication bias. Our research provides clear evidence that many published ecological and evolutionary findings are inflated. Our results highlight the importance of designing high-power empirical studies (e.g., via collaborative team science), promoting and encouraging replication studies, testing and correcting for publication bias in meta-analyses, and adopting open and transparent research practices, such as (pre)registration, data- and code-sharing, and transparent reporting. [ABSTRACT FROM AUTHOR]

We aimed to examine the bias for statistical significance using published confidence intervals in sport and exercise medicine research. Observational study. The abstracts of 48,390 articles, published in 18 sports and exercise medicine journals between 2002 and 2022, were searched using a validated text-mining algorithm that identified and extracted ratio confidence intervals (odds, hazard, and risk ratios). The algorithm identified 1744 abstracts that included ratio confidence intervals, from which 4484 intervals were extracted. After excluding ineligible intervals, the analysis used 3819 intervals, reported as 95 % confidence intervals, from 1599 articles. The cumulative distributions of lower and upper confidence limits were plotted to identify any abnormal patterns, particularly around a ratio of 1 (the null hypothesis). The distributions were compared to those from unbiased reference data, which was not subjected to p -hacking or publication bias. A bias for statistical significance was further investigated using a histogram plot of z -values calculated from the extracted 95 % confidence intervals. There was a marked change in the cumulative distribution of lower and upper bound intervals just over and just under a ratio of 1. The bias for statistical significance was also clear in a stark under-representation of z -values between − 1.96 and + 1.96, corresponding to p -values above 0.05. There was an excess of published research with statistically significant results just below the standard significance threshold of 0.05, which is indicative of publication bias. Transparent research practices, including the use of registered reports, are needed to reduce the bias in published research. [ABSTRACT FROM AUTHOR]

Abstract Background Factors contributing to the lack of understanding of research studies include poor reporting practices, such as selective reporting of statistically significant findings or insufficient methodological details. Systematic reviews have shown that prognostic factor studies continue to be poorly reported, even for important aspects, such as the effective sample size. The REMARK reporting guidelines support researchers in reporting key aspects of tumor marker prognostic studies. The REMARK profile was proposed to augment these guidelines to aid in structured reporting with an emphasis on including all aspects of analyses conducted. Methods A systematic search of prognostic factor studies was conducted, and fifteen studies published in 2015 were selected, three from each of five oncology journals. A paper was eligible for selection if it included survival outcomes and multivariable models were used in the statistical analyses. For each study, we summarized the key information in a REMARK profile consisting of details about the patient population with available variables and follow-up data, and a list of all analyses conducted. Results Structured profiles allow an easy assessment if reporting of a study only has weaknesses or if it is poor because many relevant details are missing. Studies had incomplete reporting of exclusion of patients, missing information about the number of events, or lacked details about statistical analyses, e.g., subgroup analyses in small populations without any information about the number of events. Profiles exhibit severe weaknesses in the reporting of more than 50% of the studies. The quality of analyses was not assessed, but some profiles exhibit several deficits at a glance. Conclusions A substantial part of prognostic factor studies is poorly reported and analyzed, with severe consequences for related systematic reviews and meta-analyses. We consider inadequate reporting of single studies as one of the most important reasons that the clinical relevance of most markers is still unclear after years of research and dozens of publications. We conclude that structured reporting is an important step to improve the quality of prognostic marker research and discuss its role in the context of selective reporting, meta-analysis, study registration, predefined statistical analysis plans, and improvement of marker research.

Background: Studies reporting statistically significant effect estimates tend to be more frequently published compared to studies reporting non-significant or equivalent estimates. Consequently, this may lead to distortion of the literature. The aim of this study is to assess the prevalence of reporting statistically significant effect estimates in leading oral health journals and to explore associations between the effect estimates and record characteristics. Methods: An electronic database search was undertaken of a selection of leading oral health journals including general oral health journals to identify primary oral health records published in 2019. Descriptive statistics and population average GEE logistic regression model was used to assess associations between articles reporting a statistically significant effect estimate and the record characteristics. Results: In 1335 records, 82.4% records reported a statistically significant effect estimate. All speciality journals compared to general oral health journals were less likely to publish a record with significant effect estimates. Authors based in Asia or other (OR 1.49; 95% CI :1.02,2.19; p = 0.037) were more likely to report significant effect estimates compared to those based in Europe. Interventional (OR 0.35; 0.22,0.58; p < 0.001) and observational (OR 0.56; 0.36, 0.89; p = 0.013) records were less likely to report significant effect estimates compared to in-vitro studies. Registered records were less likely to report significant effect estimates when compared to non-registered studies (OR 0.22; 95% CI :0.14,0.32; p < 0.001). Conclusion: The publishing of records with significant effect estimates is prevalent within the oral health literature. To reduce dissemination bias and overestimation of effect sizes in systematic reviews, the publishing of studies with non-significant or equivalent effect estimates should be encouraged. [ABSTRACT FROM AUTHOR]

Background: Selective outcome reporting (SOR) is a bias that occurs when the primary outcome of a randomised clinical trial (RCT) is omitted or changed. Aim: To evaluate the prevalence of SOR in RCTs on restorative treatment in primary teeth. Design: We conducted an electronic search on ClinicalTrials.gov and the World Health Organization platform (International Clinical Trials Registry Platform) on 1st of April 2021, with no registry time or language restrictions. We included RCT protocols that evaluated restorative treatments in primary teeth and excluded trials that did not have a complete publication in a scientific journal. The chi‐squared test was used to identify the association between SOR and variables as a discrepancy in the follow‐up period, the timing of registration, the type of sponsorship and the type of study design (α = 5%). Results: Of the 294 identified protocols, 30 were included in the study. 83.3% of trials were registered retrospectively. SOR was observed in 53.3% (n = 16) of the published trials and was significantly associated with a discrepancy in the follow‐up period (p =.017). Conclusions: The high prevalence of SOR in RCTs on restorative treatment proves that this is a prominent threat. A proper preregistered protocol, declaration of any protocol deviation and allowance of stakeholders to compare the protocol with that of the submitted papers will achieve transparency. [ABSTRACT FROM AUTHOR]

Objective: A recent meta‐review of attentional bias research in eating disorders suggests that meta‐analyses and systematic reviews include many low‐quality and underpowered studies (Stott et al., 2021). As such, we examined whether published research examining the link between attentional bias, using the emotional Stroop task, and eating disorders among women with eating disorders has evidential value (ruling out selective reporting of a statistically significant effect) using a p‐curve analysis. A p‐curve analysis plots statisticall significant p‐values onto a curve from.01 to.05 to examine its distribution. We hypothesized that the p‐curve would be flat, indicating no true effect. Method: The hypothesis, database search strategy, and data analytic approach were pre‐registered. The inclusion criteria were reports that compared control and eating disorder groups, reported inferential statistics, and that used body shape/weight or general threat target words. Results: Fifty published reports were included in the p‐curve analyses. Unexpectedly, the half and full p‐curves were significantly right‐skewed, indicating evidential value. However, the results were not robust to the exclusion of the seven lowest p‐values and on average, reports were underpowered. There were also 18 reports with null results (they had a p‐value greater than.05), which precluded their inclusion in the p‐curve analyses. Discussion: The findings suggest that most of the evidence from research examining attentional biases using the emotional Stroop task among women with an eating disorder or with elevated eating disorder symptoms is underpowered and so should be interpreted with considerable caution. Public Significance: Concerns have been raised about the low quality of research examining attentional biases among women with eating disorders using the emotional Stroop task. In the current research, we observed that the evidential value of primary research reporting differences between women with and without eating disorders was equivocal and had low statistical power. These results can guide researchers towards conducting more rigorous research on attentional biases among people with eating disorders. [ABSTRACT FROM AUTHOR]

A key issue in science is assessing robustness to data analysis choices, while avoiding selective reporting and providing valid inference. Specification Curve Analysis is a tool intended to prevent selective reporting. Alas, when used for inference it can create severe biases and false positives, due to wrongly adjusting for covariates, and mask important treatment effect heterogeneity. As our motivating application, it led an influential study to conclude there is no relevant association between technology use and teenager mental well‐being. We discuss these issues and propose a strategy for valid inference. Bayesian Specification Curve Analysis (BSCA) uses Bayesian Model Averaging to incorporate covariates and heterogeneous effects across treatments, outcomes and subpopulations. BSCA gives significantly different insights into teenager well‐being, revealing that the association with technology differs by device, gender and who assesses well‐being (teenagers or their parents). [ABSTRACT FROM AUTHOR]

Objectives: Reporting bias, prevalent in biomedical fields, can undermine evidence credibility. Our objective was to evaluate the proportion of discrepancies between registered protocols and published manuscripts in randomized controlled trials (RCTs) on exercise interventions for patients with chronic low back pain (CLBP)., Study Design and Setting: We conducted a cross-sectional meta-research study, starting from the 2021 "Exercise therapy for CLBP" Cochrane Review. We selected all RCTs reporting a protocol registration on a primary register of the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) or in ClinicalTrials.gov. We extracted data from both registered protocol and published manuscript of RCTs, collecting recruitment and administrative information (eg, record dates) and details of trial characteristics (eg, outcomes, arms, statistical analysis plan details [SAPs]). Independent pairs of reviewers assessed discrepancies between registered protocol and published manuscript for the reporting of primary and secondary outcomes domains, measurement instruments, time-points, number of arms and SAPs(if attached). Outcome discrepancies were characterized as addition, omission, upgrade or downgrade., Results: We included 116 RCTs reporting an available protocol registration. Overall, 100 RCTs (86.2%) distinguished between primary and secondary outcomes. Of these, 39 RCTs (39.0%) reported one or more discrepancies in primary outcomes, and 78 RCTs (78.0%) reported one or more discrepancies in secondary outcomes. Focusing on discrepancies for the primary outcome, 64.5% of added, upgraded or downgraded outcomes favored statistically significant effects. Few RCTs (n = 6) reported discrepancies in the number of arms. SAPs were poorly reported in the registered protocols (n = 3) for being compared to the publications., Conclusion: We found substantial outcome discrepancies comparing registered protocols and published manuscripts in RCTs assessing exercise interventions for patients with CLBP, with some impacting the statistical significance of the effects. Readers are encouraged to approach RCTs results in this field with caution., Competing Interests: Declaration of competing interest The authors declare that they have no competing interests., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objective: Death data from cohorts of academicians have been used to estimate pandemic excess deaths. We aimed to evaluate the validity of this approach., Study Design and Setting: Data were analyzed from living and deceased member lists from Mainland China, UK and Greece academies; and Nobel laureates (and US subset thereof). Samples of early elected academicians were probed for unrecorded deaths; datasets overtly missing deaths were excluded from further analyses. Actuarial risks were compared against the general population in the same country in respective age strata. Relative incidence risk increases in death in active pandemic periods were compared to population-wide pandemic excess death estimates for the same country., Results: Royal Society and Academy of Athens datasets overtly missed deaths. Prepandemic death rates were 4- to 12-fold lower in the Chinese Academy of Engineering (CAE) vs respective age strata of the Mainland China population. A +158% relative increase in death risk was seen in CAE data during the first 12-month of wide viral spread. Both increases (+34% in British Academy) and decreases (-27% in US Nobel laureates) in death rates occurred in pandemic (2020-22) vs prepandemic (2017-19) years; point estimates were far from known excess deaths in the respective countries (+6% and +14%, respectively). Published excess death estimates for urban-dwelling Mainland China selectively analyzed CAE that had double the pandemic death rates than another Chinese academy (Chinese Academy of Sciences)., Conclusion: Missingness, lack of representativeness, large uncertainty, and selective analysis reporting make data from academy rosters unreliable for estimating general population excess deaths., Competing Interests: Declaration of competing interest J.P.A.I. declares no conflict of interest., (Copyright © 2024 Elsevier Inc. All rights reserved.)

There is an increasing awareness that replication should become common practice in empirical studies. However, study results might fail to replicate for various reasons. The robustness of published study results can be assessed using the relatively new multiverse-analysis methodology, in which the robustness of the effect estimates against data analytical decisions is assessed. However, the uptake of multiverse analysis in empirical studies remains low, which might be due to the scarcity of guidance available on performing multiverse analysis. Researchers might experience difficulties in identifying data analytical decisions and in summarizing the large number of effect estimates yielded by a multiverse analysis. These difficulties are amplified when applying multiverse analysis to assess the robustness of the effect estimates from a mediation analysis, as a mediation analysis involves more data analytical decisions than a bivariate analysis. The aim of this paper is to provide an overview and worked example of the use of multiverse analysis to assess the robustness of the effect estimates from a mediation analysis. We showed that the number of data analytical decisions in a mediation analysis is larger than in a bivariate analysis. By using a real-life data example from the Longitudinal Aging Study Amsterdam, we demonstrated the application of multiverse analysis to a mediation analysis. This included the use of specification curves to determine the impact of data analytical decisions on the magnitude and statistical significance of the direct, indirect, and total effect estimates. Although the multiverse analysis methodology is still relatively new and future research is needed to further advance this methodology, this paper shows that multiverse analysis is a useful method for the assessment of the robustness of the direct, indirect, and total effect estimates in a mediation analysis and thereby to inform replication studies. [ABSTRACT FROM AUTHOR]

Background: Factors contributing to the lack of understanding of research studies include poor reporting practices, such as selective reporting of statistically significant findings or insufficient methodological details. Systematic reviews have shown that prognostic factor studies continue to be poorly reported, even for important aspects, such as the effective sample size. The REMARK reporting guidelines support researchers in reporting key aspects of tumor marker prognostic studies. The REMARK profile was proposed to augment these guidelines to aid in structured reporting with an emphasis on including all aspects of analyses conducted. Methods: A systematic search of prognostic factor studies was conducted, and fifteen studies published in 2015 were selected, three from each of five oncology journals. A paper was eligible for selection if it included survival outcomes and multivariable models were used in the statistical analyses. For each study, we summarized the key information in a REMARK profile consisting of details about the patient population with available variables and follow-up data, and a list of all analyses conducted. Results: Structured profiles allow an easy assessment if reporting of a study only has weaknesses or if it is poor because many relevant details are missing. Studies had incomplete reporting of exclusion of patients, missing information about the number of events, or lacked details about statistical analyses, e.g., subgroup analyses in small populations without any information about the number of events. Profiles exhibit severe weaknesses in the reporting of more than 50% of the studies. The quality of analyses was not assessed, but some profiles exhibit several deficits at a glance. Conclusions: A substantial part of prognostic factor studies is poorly reported and analyzed, with severe consequences for related systematic reviews and meta-analyses. We consider inadequate reporting of single studies as one of the most important reasons that the clinical relevance of most markers is still unclear after years of research and dozens of publications. We conclude that structured reporting is an important step to improve the quality of prognostic marker research and discuss its role in the context of selective reporting, meta-analysis, study registration, predefined statistical analysis plans, and improvement of marker research. [ABSTRACT FROM AUTHOR]

Undisclosed discrepancies often exist between study registrations and their associated publications. Discrepancies can increase risk of bias, and when undisclosed, they disguise this increased risk of bias from readers. To remedy this issue, we developed an intervention called discrepancy review. We provided journals with peer reviewers specifically assigned to check for undisclosed discrepancies between registrations and manuscripts submitted to journals. We performed discrepancy review on 18 manuscripts submitted to Nicotine and Tobacco Research and three manuscripts submitted to the European Journal of Personality. We iteratively refined the discrepancy review process based on feedback from discrepancy reviewers, editors and authors. Authors addressed the majority of discrepancy reviewer comments, and there was no opposition to running a trial from authors, editors or discrepancy reviewers. Outcome measures for a trial of discrepancy review could include the presence of primary or secondary outcome discrepancies, whether publications that are not the primary report from a clinical trial registration are clearly described as such, whether registrations are permanent, and an overarching subjective assessment of the impact of discrepancies in published articles. We found that discrepancy review could feasibly be introduced as a regular practice at some journals interested in this process. A full trial of discrepancy review would be needed to evaluate its impact on reducing undisclosed discrepancies.

BACKGROUND: Selective reporting of trial results is common. OBJECTIVE: To study selective reporting in clinical study reports, company trial registers and publications of quality of life in placebo-controlled trials of antidepressants. METHODS: We compared clinical study reports of four antidepressants (fluoxetine, duloxetine, paroxetine and sertraline) obtained from two European drug regulators, data from online company registers, and publications received or retrieved from Eli Lilly and GlaxoSmithKline. Pfizer was also contacted but did not provide any publications. RESULTS: We included 15 trials (19,015 pages) and 4717 patients. Six trials had used SF-36, seven EQ-5D and two both instruments. Nine of the 15 CSRs (60%) displayed selective reporting. In the companies' online registers, there was selective reporting for all 15 trials (100%). We received 20 publications from Eli Lilly and retrieved six from the GlaxoSmithKline register. There was selective reporting in 24 of the 26 publications (92%). Despite extensive selective reporting, we found only small differences between placebo and active drugs. CONCLUSIONS: Access to the full raw data from clinical trials and to case report forms for all patients are needed to evaluate the effect of antidepressants on quality of life. Regulatory agencies should refuse to approve drugs or new indications based on incomplete reporting. [ABSTRACT FROM AUTHOR]

Selective reporting of antibiotic susceptibility test results may help guide appropriate antibiotic prescribing, particularly for urinary tract infections. Our objective was to describe laboratory urine culture susceptibility reporting practices and to estimate their impact on antibiotic prescribing in outpatients. We examined all positive urine cultures with Escherichia coli , Klebsiella pneumoniae , or Proteus mirabilis associated with an antibiotic prescription among outpatients over 65 years of age in Ontario, Canada from 2014 through 2017. We evaluated antibiotic prescribing in the empirical window (1–3 days before culture result) and in the directed window (0–5 days after culture result). Unadjusted and adjusted odds ratios were reported to estimate the association between reporting and prescribing. In total 113 780 eligible urine cultures from 48 laboratories were included in the study cohort. Susceptibility reporting practices were highly variable between laboratories, with a range across antibiotics from norfloxacin (n = 5/48, 10.4% reporting) to nitrofurantoin (n = 40/48, 83.3% reporting). Reporting antibiotic susceptibility was associated with increased odds of prescribing that antibiotic in the directed window (aOR 2.98, 95%CI 2.07–4.28). At the laboratory level, the proportion of urine cultures reporting specific antibiotic susceptibility results was also associated with an increase in prescribing of that antibiotic in the empirical window (adjusted OR 1.23, 95%CI 1.13–1.33, per 25% increase in reporting). Laboratory reporting of antibiotic susceptibility results for urine cultures is associated with empirical and directed prescribing of the reported antibiotics. Laboratories can play an important role in guiding appropriate antibiotic selection for urinary indications. [ABSTRACT FROM AUTHOR]

Selective publication and reporting in individual papers compromise the scientific record, but are meta‐analyses as compromised as their constituent studies? We systematically sampled 63 meta‐analyses (each comprising at least 40 studies) in PLoS One, top medical journals, top psychology journals, and Metalab, an online, open‐data database of developmental psychology meta‐analyses. We empirically estimated publication bias in each, including only the peer‐reviewed studies. Across all meta‐analyses, we estimated that "statistically significant" results in the expected direction were only 1.17 times more likely to be published than "nonsignificant" results or those in the unexpected direction (95% CI: [0.93, 1.47]), with a confidence interval substantially overlapping the null. Comparable estimates were 0.83 for meta‐analyses in PLoS One, 1.02 for top medical journals, 1.54 for top psychology journals, and 4.70 for Metalab. The severity of publication bias did differ across individual meta‐analyses; in a small minority (10%; 95% CI: [2%, 21%]), publication bias appeared to favor "significant" results in the expected direction by more than threefold. We estimated that for 89% of meta‐analyses, the amount of publication bias that would be required to attenuate the point estimate to the null exceeded the amount of publication bias estimated to be actually present in the vast majority of meta‐analyses from the relevant scientific discipline (exceeding the 95th percentile of publication bias). Study‐level measures ("statistical significance" with a point estimate in the expected direction and point estimate size) did not indicate more publication bias in higher‐tier versus lower‐tier journals, nor in the earliest studies published on a topic versus later studies. Overall, we conclude that the mere act of performing a meta‐analysis with a large number of studies (at least 40) and that includes non‐headline results may largely mitigate publication bias in meta‐analyses, suggesting optimism about the validity of meta‐analytic results. [ABSTRACT FROM AUTHOR]

Background: Assessments of scientific evidence often involve a systematic comparison of findings within and across studies. An important consideration in systematic reviews is multiplicity, which can arise from simultaneous consideration of multiple independent and dependent variables, use of multiple statistical models, or multiple subgroup analyses. Multiplicity can affect the results of evidence assessment, if accompanied by incomplete or selective reporting. Birth cohort studies investigating prenatal/neonatal exposure to polychlorinated biphenyls (PCBs) and their relation to neurodevelopmental measures during follow up offer an interesting opportunity for assessing multiplicity and completeness of reporting because the literature on the subject is voluminous, and the data typically allow considerable flexibility in terms of choice of analysis and reporting of findings. Methods: Following a systematic search, each relevant publication was characterized with respect to its methods of exposure assessment, outcome characterization, analysis and reporting. Based on the total number of unique exposure and outcome categories we calculated the number of possible exposure-outcome associations that could have been examined within and across cohorts. Each association was categorized as “reported” or “not reported/not evaluated”, and the number of studies that reported each association was ascertained. Results: A total of 208 prenatal/neonatal exposures and 461 outcomes were measured across 34 cohorts and 111 publications. Only 29 associations were presented in at least three studies; of those, only 21 associations were reported within the same age group allowing a meaningful side-by-side comparison. Percentage of within-cohort associations that were reported among all those that could have been reported based on the available data ranged from 6% to 100%. Conclusions: The literature on PCBs and neurodevelopment exemplifies a situation wherein despite large numbers of published studies, inconsistent or incomplete reporting of multiple results impede systematic reviews.

In their book 'Nudge: Improving Decisions About Health, Wealth and Happiness', Thaler & Sunstein (2009) argue that choice architectures are promising public policy interventions. This research programme motivated the creation of 'nudge units', government agencies which aim to apply insights from behavioural science to improve public policy. We closely examine a meta-analysis of the evidence gathered by two of the largest and most influential nudge units (DellaVigna & Linos (2022 Econometrica , 81-116 (doi:10.3982/ECTA18709))) and use statistical techniques to detect reporting biases. Our analysis shows evidence suggestive of selective reporting. We additionally evaluate the public pre-analysis plans from one of the two nudge units (Office of Evaluation Sciences). We identify several instances of excellent practice; however, we also find that the analysis plans and reporting often lack sufficient detail to evaluate (unintentional) reporting biases. We highlight several improvements that would enhance the effectiveness of the pre-analysis plans and reports as a means to combat reporting biases. Our findings and suggestions can further improve the evidence base for policy decisions.90 , 81-116 (doi:10.3982/ECTA18709))) and use statistical techniques to detect reporting biases. Our analysis shows evidence suggestive of selective reporting. We additionally evaluate the public pre-analysis plans from one of the two nudge units (Office of Evaluation Sciences). We identify several instances of excellent practice; however, we also find that the analysis plans and reporting often lack sufficient detail to evaluate (unintentional) reporting biases. We highlight several improvements that would enhance the effectiveness of the pre-analysis plans and reports as a means to combat reporting biases. Our findings and suggestions can further improve the evidence base for policy decisions., Competing Interests: We declare we have no competing interests., (© 2024 The Authors.)

Background: Selective reporting is a promising tool for antimicrobial stewardship, but in wound cultures, its effects on the use of antimicrobials are unknown. Our HUS Diagnostic Center Bacteriology laboratory refined its selective reporting protocol for wound cultures during 2017-2018. In this study we aimed to show our protocol's impact on the frequency of antimicrobial escalation., Methods: We performed a retrospective cohort study of patients in the wound-care ward of a primary-care hospital in Helsinki, Finland, from 2014 to 2016 (pre-intervention) and from 2019 to April 2021 (post-intervention). With the inclusion criterion being wound-culture collection, this provided us with 299 patients, of which 152 were in the pre-intervention group, and 147 were post-intervention. We collected the data from medical records and compared the pre-intervention- with the post-intervention group in terms of patient profiles, microbiology reports, antimicrobial treatment, and treatment outcomes., Findings: In the pre-intervention group 40% of the patients were male and 60% female and in the post-intervention group 49% and 51% respectively. The frequency of AST reported had decreased from 63% in the pre-intervention group to 37% post-intervention (OR 0.35, p < 0.001). The post-intervention group demonstrated lower frequencies of antimicrobial treatment 7 d after wound culture collection, 82% pre-intervention vs 58% post-intervention (OR 0.31, p < 0.001), and antimicrobial escalation, 42% vs 20% (OR 0.35, p < 0.001) respectively. Length of hospital stay, and all-cause mortality were similar between the groups., Interpretation: Selective reporting of wound cultures appears an effective and safe measure to reduce the use of antimicrobials., Funding: HUS Diagnostic Center., Competing Interests: Declaration of interests We declare no competing interests., (Copyright © 2024 The Author(s). Published by Elsevier B.V. All rights reserved.)

Abstract Clinical trials and systematic reviews of clinical trials inform healthcare decisions. There is growing concern, however, about results from clinical trials that cannot be reproduced. Reasons for nonreproducibility include that outcomes are defined in multiple ways, results can be obtained using multiple methods of analysis, and trial findings are reported in multiple sources (“multiplicity”). Multiplicity combined with selective reporting can influence dissemination of trial findings and decision-making. In particular, users of evidence might be misled by exposure to selected sources and overly optimistic representations of intervention effects. In this commentary, drawing from our experience in the Multiple Data Sources in Systematic Reviews (MUDS) study and evidence from previous research, we offer practical recommendations to enhance the reproducibility of clinical trials and systematic reviews.

Abstract Background Registration of clinical trials is critical for promoting transparency and integrity in medical research; however, trials must be registered in a prospective fashion to deter unaccounted protocol modifications or selection of alternate outcomes that may enhance favorability of reported findings. We assessed adherence to the International Committee of Medical Journal Editors’ (ICMJE) prospective registration policy and identified the frequency of registrations occurring after potential observation of primary outcome data among trials published in the highest-impact journals associated with US professional medical societies. Additionally, we examined whether trials that are unregistered or registered after potential observation of primary outcome data were more likely to report favorable findings. Methods We conducted a retrospective, cross-sectional analysis of the 50 most recently published clinical trials that reported primary results in each of the ten highest-impact US medical specialty society journals between 1 January 2010 and 31 December 2015. We used descriptive statistics to characterize the proportions of trials that were: registered; registered retrospectively; registered retrospectively potentially after initial ascertainment of primary outcomes; and reporting favorable findings, overall and stratified by journal and trial characteristics. Chi-squared analyses were performed to assess differences in registration by journal and trial characteristics. Results We reviewed 6869 original research reports published between 1 January 2010 and 31 December 2015 to identify a total of 486 trials across 472 publications. Of these 486 trials, 47 (10%) were unregistered. Among 439 registered trials, 340 (77%) were registered prospectively and 99 (23%) retrospectively. Sixty-seven (68%) of these 99 retrospectively registered trials, or 15% of all 439 registered trials, were registered after potential observation of primary outcome data ascertained among participants enrolled at inception. Industry-funded trials, those with enrollment sites in the US, as well as those assessing FDA-regulated interventions each had lower rates of retrospective registration. Unregistered trials were more likely to report favorable findings than were registered trials (89% vs. 64%; relative risk (RR) = 1.38, 95% confidence interval (CI) = 1.20–1.58; p = 0.004), irrespective of registration timing. Conclusions Adherence to the ICMJE’s prospective registration policy remains sub-standard, even in the highest-impact journals associated with US professional medical societies. These journals frequently published unregistered trials and trials registered after potential observation of primary outcome data.

Abstract Background Inadequate planning, selective reporting, and incomplete reporting of outcomes in randomized controlled trials (RCTs) contribute to the problem of waste of research. We aimed to describe such a waste and to examine to what extent this waste could be avoided. Methods This research-on-research study was based on RCTs included in Cochrane reviews with a summary of findings (SoF) table. We considered the outcomes reported in the SoF tables as surrogates for important outcomes for patients and other decision makers. We used a three-step approach. (1) First, in each review, we identified, for each important outcome, RCTs that were excluded from the corresponding meta-analysis. (2) Then, for these RCTs, we systematically searched for registrations and protocols to distinguish between inadequate planning (an important outcome was not reported in registries or protocols), selective reporting (an important outcome was reported in registries or protocols but not in publications), and incomplete reporting (an important outcome was incompletely reported in publications). (3) Finally, we assessed, with the consensus of five experts, the feasibility and cost of measuring the important outcomes that were not planned. We considered inadequately planned or selectively or incompletely reported important outcomes as avoidable waste if the outcome could have been easily measured at no additional cost based on expert evaluation. Results Of the 2711 RCTs included in the main comparison of 290 reviews, 2115 (78%) were excluded from at least one meta-analysis of important outcomes. Every trial contributed to 55%, on average, of the meta-analyses of important outcomes. Of the 310 RCTs published in 2010 or later, 156 were registered. Inadequate planning affected 79% of these RCTs, whereas incomplete and selective reporting affected 41% and 15%, respectively. For 63% of RCTs, we found at least one missing important outcome for which the waste was avoidable and for 30%, the waste was avoidable for all important outcomes. Conclusions Most of the RCTs included in our sample did not contribute to all the important outcomes in meta-analyses, mostly because of inadequate planning or incomplete reporting. A large part of this waste of research seemed to be avoidable.

Abstract Background Recognizing the value of promoting public access to clinical trial protocols, Trials pioneered the way for their publication over a decade ago. However, despite major advances in the public accessibility of information about trial methods and results, protocol sharing remains relatively rare. Main body Protocol sharing facilitates the critical appraisal of clinical trials and helps to identify and deter the selective reporting of outcomes and analyses. Challenges to the routine availability of high quality trial protocols include the gaps in incentives and adherence mechanisms, limited venues for sharing the original and final protocol versions, and the need for mechanisms to ensure transparent and complete protocol content. Conclusions We propose recommendations for addressing key challenges to protocol sharing in order to promote routine public access to protocols for the benefit of patients and other users of evidence from clinical trials.

Abstract Background Prospective study protocols and registrations can play a significant role in reducing incomplete or selective reporting of primary biomedical research, because they are pre-specified blueprints which are available for the evaluation of, and comparison with, full reports. However, inconsistencies between protocols or registrations and full reports have been frequently documented. In this systematic review, which forms part of our series on the state of reporting of primary biomedical, we aimed to survey the existing evidence of inconsistencies between protocols or registrations (i.e., what was planned to be done and/or what was actually done) and full reports (i.e., what was reported in the literature); this was based on findings from systematic reviews and surveys in the literature. Methods Electronic databases, including CINAHL, MEDLINE, Web of Science, and EMBASE, were searched to identify eligible surveys and systematic reviews. Our primary outcome was the level of inconsistency (expressed as a percentage, with higher percentages indicating greater inconsistency) between protocols or registration and full reports. We summarized the findings from the included systematic reviews and surveys qualitatively. Results There were 37 studies (33 surveys and 4 systematic reviews) included in our analyses. Most studies (n = 36) compared protocols or registrations with full reports in clinical trials, while a single survey focused on primary studies of clinical trials and observational research. High inconsistency levels were found in outcome reporting (ranging from 14% to 100%), subgroup reporting (from 12% to 100%), statistical analyses (from 9% to 47%), and other measure comparisons. Some factors, such as outcomes with significant results, sponsorship, type of outcome and disease speciality were reported to be significantly related to inconsistent reporting. Conclusions We found that inconsistent reporting between protocols or registrations and full reports of primary biomedical research is frequent, prevalent and suboptimal. We also identified methodological issues such as the need for consensus on measuring inconsistency across sources for trial reports, and more studies evaluating transparency and reproducibility in reporting all aspects of study design and analysis. A joint effort involving authors, journals, sponsors, regulators and research ethics committees is required to solve this problem.

Clinical research should conform to high standards of ethical and scientific integrity, given that human lives are at stake. However, economic incentives can generate conflicts of interest for investigators, who may be inclined to withhold unfavorable results or even tamper with data in order to achieve desired outcomes. To shed light on the integrity of clinical trial results, this paper systematically analyzes the distribution of P values of primary outcomes for phase II and phase III drug trials reported to the ClinicalTrials.gov registry. First, we detect no bunching of results just above the classical 5% threshold for statistical significance. Second, a density-discontinuity test reveals an upward jump at the 5% threshold for phase III results by small industry sponsors. Third, we document a larger fraction of significant results in phase III compared to phase II. Linking trials across phases, we find that early favorable results increase the likelihood of continuing into the next phase. Once we take into account this selective continuation, we can explain almost completely the excess of significant results in phase III for trials conducted by large industry sponsors. For small industry sponsors, instead, part of the excess remains unexplained. [ABSTRACT FROM AUTHOR]

Introduction: Selective reporting of antibiotic susceptibility testing (AST) results is a potentially interesting tool for antibiotic stewardship. It consists of performing AST according to usual practices, but the results are reported to the prescriber only for a few antibiotics (i.e. first-line agents) or not reported at all when colonization is likely.Areas covered: We retrieved 20 studies exploring the impact of selective reporting. Overall, selective reporting is able to influence antibiotic use, both discouraging prescription in case of colonization, and promoting the selection of narrow-spectrum agents. Most studies concerned urine samples. Evidence on the impact on antibiotic resistance is insufficient. Unintended consequences were not observed, but evidence on this topic is scarce. Selective reporting is well implemented in a few countries, and a huge heterogeneity of practices exists.Expert opinion: Evidence shows that selective reporting can help reducing inappropriate and unnecessary antibiotic prescriptions. Uncomplicated urinary tract infections are probably the best initial target, both in hospital and community settings, but other non-severe infections can be a suitable option. The implementation of selective reporting should be promoted by the scientific community, with detailed practical guidelines, and its impact should be further assessed in large interventional studies. [ABSTRACT FROM AUTHOR]

Abstract Background There is evidence that direct journal endorsement of reporting guidelines can lead to important improvements in the quality and reliability of the published research. However, over the last 20 years, there has been a proliferation of reporting guidelines for different study designs, making it impractical for a journal to explicitly endorse them all. The objective of this study was to investigate whether a decision tree tool made available during the submission process facilitates author identification of the relevant reporting guideline. Methods This was a prospective 14-week before–after study across four speciality medical research journals. During the submission process, authors were prompted to follow the relevant reporting guideline from the EQUATOR Network and asked to confirm that they followed the guideline (‘before’). After 7 weeks, this prompt was updated to include a direct link to the decision-tree tool and an additional prompt for those authors who stated that ‘no guidelines were applicable’ (‘after’). For each article submitted, the authors’ response, what guideline they followed (if any) and what reporting guideline they should have followed (including none relevant) were recorded. Results Overall, 590 manuscripts were included in this analysis—300 in the before cohort and 290 in the after. There were relevant reporting guidelines for 75% of manuscripts in each group; STROBE was the most commonly applicable reporting guideline, relevant for 35% (n = 106) and 37% (n = 106) of manuscripts, respectively. Use of the tool was associated with an 8.4% improvement in the number of authors correctly identifying the relevant reporting guideline for their study (p