Search

Rationale Aims and Objectives: As the recent literature has growing concerns about research replicability and the misuse and misconception of P-values, the fragility index (FI) has been an attractive measure to assess the robustness (or fragility) of clinical study results with binary outcomes. It is defined as the minimum number of event status modifications that can alter a study result's statistical significance (or non-significance). Owing to its intuitive concept, the FI has been applied to assess the fragility of clinical studies of various specialties. However, the FI may be limited in certain settings. As a relatively new measure, more work is needed to examine its properties., Methods: This article explores several factors that may impact the derivation of the FI, including how event status is modified and the impact of significance levels. Moreover, we propose novel methods to visualize the fragility of a study's result. These factors and methods are illustrated using worked examples of artificial datasets. Randomized controlled trials on antidepressant drugs are also used to evaluate their real-world performance., Results: The FI depends on the treatment arm(s) in which event status is modified, whether the original study result is significant, the statistical method used for calculating the P-value, and the threshold for determining statistical significance. Also, the proposed visualization methods can clearly demonstrate a study result's fragility, which may be useful supplements to the single value of the FI., Conclusions: Our findings may help clinicians properly use the FI and appraise the reliability of a study's conclusion., (© 2020 John Wiley & Sons, Ltd.)

Motivated by the recent replication and reproducibility crisis, Gelman and Carlin (2014, Perspect. Psychol. Sci., 9, 641) advocated focusing on controlling for Type S/M errors, instead of the classic Type I/II errors, when conducting hypothesis testing. In this paper, we aim to fill several theoretical gaps in the methodology proposed by Gelman and Carlin (2014, Perspect. Psychol. Sci., 9, 641). In particular, we derive the closed-form expression for the expected Type M error, and study the mathematical properties of the probability of Type S error as well as the expected Type M error, such as monotonicity. We demonstrate the advantages of our results through numerical and empirical examples., (© 2018 The British Psychological Society.)

The association of biomedical sciences with mathematics is inextricable and undeniable. It should be considered that gaining confidence or strong belief within the framework of a scientific experiment - in the 'compulsory' marriage with statistics - does not mean total and undisturbed certainty. The null hypothesis significance testing (NHST) has been the kernel of statistical data analysis for a long period. Medicine is making use of it in a very wide range. The main problem in NHST arises out of a dichotomic perception of reality. Scientific reasoning was biased in the aftermath of the uncritical, almost dogmatic, trust in a level of statistical significance, namely a p-value, defined as 'the dictating paradigm of p-value'. The level of statistical significance (p-value) is not completely plausible and objective; it cannot be equated with the nature of a phenomenon or population characteristics. The p-value refers directly to a formulated hypothesis only, that is the probability of observing a given outcome under the condition posited by a specific null hypothesis and given a specific model of the distribution of outcomes under the null hypothesis. Experts claim that unjustified conclusions and guidelines have become the bane of many scientific journals. Investigators should be able to interpret study results in a broad context and look at them critically. Statistics indeed is a leaven to further scientific discussion, not its conclusion. It is not easy to abandon the p-value. Lots of researchers recommended an ecumenical approach to the statistical analysis of research data, for example analyses of the same results using a variety of statistical procedures. Many of them express the view that a threshold level of statistical significance should be lower than p < 0.05. To enhance the plausibility and true scientific value of research works, investigators should also take into consideration the effect size along with its confidence limits. The effect size is a kind of added value and should be determined a priori. Account must be taken of their origin, conditioning (medical knowledge), as well as specific implications in clinical practice. The effect size makes possible to compare results of various research works. It is argued that to the rescue for modern science can come research works that bear the stamp of reproducibility, for example studies conducted in several independent centres, based on coherent and meticulous methodology, or data from various centres typed in one spotless big database and submitted to rigorous and deliberate multilevel statistical analyses., (© 2018 Acta Ophthalmologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.)

Objectives: The Fragility Index (FI) is a novel tool for evaluating the robustness of statistically significant findings in a randomized control trial (RCT). It measures the number of events upon which statistical significance depends. We sought to calculate the FI scores for RCTs in the head and neck cancer literature where surgery was a primary intervention., Data Sources: Potential articles were identified in PubMed (MEDLINE), Embase, and Cochrane without publication date restrictions., Review Methods: Two reviewers independently screened eligible RCTs reporting at least one dichotomous and statistically significant outcome. The data from each trial were extracted and the FI scores were calculated. Associations between trial characteristics and FI were determined., Results: In total, 27 articles were identified. The median sample size was 67.5 (interquartile range [IQR] = 42-143) and the median number of events per trial was 8 (IQR = 2.25-18.25). The median FI score was 1 (IQR = 0-2.5), meaning that changing one patient from a nonevent to an event in the treatment arm would change the result to a statistically nonsignificant result, or P > .05. The FI score was less than the number of patients lost to follow-up in 71% of cases. The FI score was found to be moderately correlated with P value (ρ = -0.52, P = .007) and with journal impact factor (ρ = 0.49, P = .009) on univariable analysis. On multivariable analysis, only the P value was found to be a predictor of FI score (P = .001)., Conclusions: Randomized trials in the head and neck cancer literature where surgery is a primary modality are relatively nonrobust statistically with low FI scores. Laryngoscope, 128:2094-2100, 2018., (© 2018 The American Laryngological, Rhinological and Otological Society, Inc.)

There is a tendency to automatically use two-sided tests to assess the statistical significance of experimental results. Yet if a theory predicts the direction of an experimental outcome, or if for some practical (eg clinical) reason an outcome in that direction is the only one of interest, then it makes sense to use a one-sided test. The use of a two-sided test in these situations will lead to too many false negatives. Consequently treatment effects that corroborate a theory or that are of practical importance may be missed. This problem becomes particularly acute in the case of borderline results. Following a nonsignificant one-sided test, the possibility of an effect in the direction opposite to that predicted or required can be assessed in an exploratory fashion by computing the odds in favour of such an effect. Anyone is then at liberty to pursue this possibility as they see fit. The question of whether to use a one-sided or two-sided statistical test should always be decided on logical grounds not statistical ones, and suspicions regarding the motives of the investigator(s) should be disregarded. On the other hand, this choice can be avoided altogether by assuming that a treatment always has some effect (however small) and then computing the strength of the evidence in favour of the observed or predicted/required effect (ie 1-P, where P is the one-sided significance level of the test). With this approach one-sided and two-sided tests yield identical results, and so there is effectively only one type of test., (© 2017 John Wiley & Sons Australia, Ltd.)