Search

This primer by the Center for Reproducible Science gives a first taste of "Dynamic reporting": What it is? Why it is useful? and How you can use it. &nbsp

First update of the preprint

We present a novel and easy-to-use method for calibrating error-rate based confidence intervals to evidence-based support intervals. Support intervals are obtained from inverting Bayes factors based on a parameter estimate and its standard error. A $k$ support interval can be interpreted as "the observed data are at least $k$ times more likely under the included parameter values than under a specified alternative". Support intervals depend on the specification of prior distributions for the parameter under the alternative, and we present several types that allow different forms of external knowledge to be encoded. We also show how prior specification can to some extent be avoided by considering a class of prior distributions and then computing so-called minimum support intervals which, for a given class of priors, have a one-to-one mapping with confidence intervals. We also illustrate how the sample size of a future study can be determined based on the concept of support. Finally, we show how the bound for the type I error rate of Bayes factors leads to a bound for the coverage of support intervals. An application to data from a clinical trial illustrates how support intervals can lead to inferences that are both intuitive and informative.

Power priors are used for incorporating historical data in Bayesian analyses by taking the likelihood of the historical data raised to the power $\alpha$ as the prior distribution for the model parameters. The power parameter $\alpha$ is typically unknown and assigned a prior distribution, most commonly a beta distribution. Here, we give a novel theoretical result on the resulting marginal posterior distribution of $\alpha$ in case of the the normal and binomial model. Counterintuitively, when the current data perfectly mirror the historical data and the sample sizes from both data sets become arbitrarily large, the marginal posterior of $\alpha$ does not converge to a point mass at $\alpha = 1$ but approaches a distribution that hardly differs from the prior. The result implies that a complete pooling of historical and current data is impossible if a power prior with beta prior for $\alpha$ is used.

This release includes all relevant files to reproduce the arXiv preprint (https://arxiv.org/abs/2203.13076).

Comparative simulation studies are workhorse tools for benchmarking statistical methods. As with other empirical studies, the success of simulation studies hinges on the quality of their design, execution and reporting. If not conducted carefully and transparently, their conclusions may be misleading. In this paper we discuss various questionable research practices which may impact the validity of simulation studies, some of which cannot be detected or prevented by the current publication process in statistics journals. To illustrate our point, we invent a novel prediction method with no expected performance gain and benchmark it in a pre-registered comparative simulation study. We show how easy it is to make the method appear superior over well-established competitor methods if questionable research practices are employed. Finally, we provide concrete suggestions for researchers, reviewers and other academic stakeholders for improving the methodological quality of comparative simulation studies, such as pre-registering simulation protocols, incentivizing neutral simulation studies and code and data sharing.

It is now widely accepted that the standard inferential toolkit used by the scientific research community-null-hypothesis significance testing (NHST)-is not fit for purpose. Yet despite the threat posed to the scientific enterprise, there is no agreement concerning alternative approaches for evidence assessment. This lack of consensus reflects long-standing issues concerning Bayesian methods, the principal alternative to NHST. We report on recent work that builds on an approach to inference put forward over 70 years ago to address the well-known "Problem of Priors" in Bayesian analysis, by reversing the conventional prior-likelihood-posterior ("forward") use of Bayes' theorem. Such Reverse-Bayes analysis allows priors to be deduced from the likelihood by requiring that the posterior achieve a specified level of credibility. We summarise the technical underpinning of this approach, and show how it opens up new approaches to common inferential challenges, such as assessing the credibility of scientific findings, setting them in appropriate context, estimating the probability of successful replications, and extracting more insight from NHST while reducing the risk of misinterpretation. We argue that Reverse-Bayes methods have a key role to play in making Bayesian methods more accessible and attractive for evidence assessment and research synthesis. As a running example we consider a recently published meta-analysis from several randomised controlled trials (RCTs) investigating the association between corticosteroids and mortality in hospitalised patients with COVID-19.

Eric Gibson is to be congratulated for a thoughtful review of the role of p-values in the assessment of the strength of evidence of research findings in pharmaceutical drug development. This perspe...

If a scientific study reports a discovery with a p-value at or around 0.05, how credible is it? And what are the chances that a replication of this study will produce a similarly “significant” finding? Leonhard Held, Samuel Pawel and Simon Schwab's answers may surprise you

Infection with the hepatitis E virus (HEV) is one of the main causes of acute hepatitis worldwide. Given that, the histopathology of hepatitis E is relatively poorly characterized, and it is unclear what exactly determines its remarkable variability. The aim of our study was a systematic analysis of hepatitis E histology, especially with regard to the clinical setting. Fifty-two liver samples (48 biopsies, 1 liver explant, 3 autopsy livers) from 41 patients with molecularly proven hepatitis E (28 HEV genotype (gt) 3, three gt 1, one gt 4 and 9 undetermined gt) were systematically evaluated for 33 histopathologic features. Following one approach, the biopsies were assigned to one of five generic histologic patterns. In another approach, they were subjected to hierarchical clustering. We found that 23/41 (56%) patients were immunocompromised, whereas 18 (44%) had no known immunosuppression. Five patients (12%) had pre-existing liver disease (LD). The histopathologic spectrum ranged from almost normal to acute, chronic, and steato-hepatitis to subtotal necrosis, and was thus distributed across all five generic patterns. Hierarchical clustering, however, identified three histopathologic clusters (C1-C3), which segregated along the immune status and pre-existing LD: C1 comprised mostly patients with pre-existing LD; histology mainly reflected the respective LD without pointing to the additional hepatitis E. C2 comprised mostly immunocompetent patients; histology mainly displayed florid hepatitis. C3 comprised mostly immunocompromised patients; histology mainly displayed smoldering hepatitis. Accordingly, C1-C3 differed markedly with respect to their clinical and histopathologic differential diagnoses. Hierarchical clustering suggests three groups with distinct histopathologies, indicating biologically different manifestations of hepatitis E. The association of histopathologic changes with the patient's immune status and pre-existing LD plausibly explains the diversity of hepatitis E histopathology, and suggests that these factors are the crucial underlying determinants. We expect our results to improve patient management by guiding the clinico-pathologic diagnosis of hepatitis E.