Search

Magnitude-based inference (MBI) is a controversial statistical method that has been used in hundreds of papers in sports science despite criticism from statisticians. To better understand how this method has been applied in practice, we systematically reviewed 232 papers that used MBI. We extracted data on study design, sample size, and choice of MBI settings and parameters. Median sample size was 10 per group (interquartile range, IQR: 8-15) for multi-group studies and 14 (IQR: 10-24) for single-group studies; few studies reported a priori sample size calculations (15%). Authors predominantly applied MBI's default settings and chose "mechanistic/non-clinical" rather than "clinical" MBI even when testing clinical interventions (only 16 studies out of 232 used clinical MBI). Using these data, we can estimate the Type I error rates for the typical MBI study. Authors frequently made dichotomous claims about effects based on the MBI criterion of a "likely" effect and sometimes based on the MBI criterion of a "possible" effect. When the sample size is n = 8 to 15 per group, these inferences have Type I error rates of 12%-22% and 22%-45%, respectively. High Type I error rates were compounded by multiple testing: Authors reported results from a median of 30 tests related to outcomes; and few studies specified a primary outcome (14%). We conclude that MBI has promoted small studies, promulgated a "black box" approach to statistics, and led to numerous papers where the conclusions are not supported by the data. Amidst debates over the role of p-values and significance testing in science, MBI also provides an important natural experiment: we find no evidence that moving researchers away from p-values or null hypothesis significance testing makes them less prone to dichotomization or over-interpretation of findings.

Ridinilazole, a novel targeted antibacterial being developed for the treatment of C. difficile infection (CDI) and prevention of recurrence, was shown in a recent Phase 2 study to be superior to vancomycin with regard to the primary efficacy measure, sustained clinical response (SCR), with the superiority being driven primarily by marked reductions in the rates of CDI recurrence within 30 days. Tolerability of ridinilazole was comparable to that of vancomycin. The current nested cohort study compared the effects of ridinilazole and vancomycin on fecal microbiota during and after treatment among participants in the Phase 2 study. Changes in the microbiota were assessed using qPCR and high-throughput sequencing on participants' stools collected at multiple time-points (baseline [Day 1], Day 5, end-of-treatment [EOT; Day 10], Day 25, end-of-study [EOS; Day 40], and at CDI recurrence). qPCR analyses showed profound losses of Bacteroides, C. coccoides, C. leptum, and Prevotella groups at EOT with vancomycin treatment, while ridinilazole-treated participants had a modest decrease in C. leptum group levels at EOT, with levels recovering by Day 25. Vancomycin-treated participants had a significant increase in the Enterobacteriaceae group, with this increase persisting beyond EOT. At EOT, alpha diversity decreased with both antibiotics, though to a significantly lesser extent with ridinilazole (p

Behavioral responses to odors rely first upon their accurate detection by peripheral sensory organs followed by subsequent processing within the brain's olfactory system and higher centers. These processes allow the animal to form a unified impression of the odor environment and recognize combinations of odorants as single entities. To investigate how interactions between peripheral and central olfactory pathways shape odor perception, we transplanted antennal imaginal discs between larval males of two species of moth Heliothis virescens and Heliothis subflexa that utilize distinct pheromone blends. During metamorphic development olfactory receptor neurons originating from transplanted discs formed connections with host brain neurons within olfactory glomeruli of the adult antennal lobe. The normal antennal receptor repertoire exhibited by males of each species reflects the differences in the pheromone blends that these species employ. Behavioral assays of adult transplant males revealed high response levels to two odor blends that were dissimilar from those that attract normal males of either species. Neurophysiological analyses of peripheral receptor neurons and central olfactory neurons revealed that these behavioral responses were a result of: 1. the specificity of H. virescens donor olfactory receptor neurons for odorants unique to the donor pheromone blend and, 2. central odor recognition by the H. subflexa host brain, which typically requires peripheral receptor input across 3 distinct odor channels in order to elicit behavioral responses.

In a recent individual patient data meta-analysis, acupuncture was found to be superior to both sham and non-sham controls in patients with chronic pain. In this paper we identify variations in types of sham and non-sham controls used and analyze their impact on the effect size of acupuncture.Based on literature searches of acupuncture trials involving patients with headache and migraine, osteoarthritis, and back, neck and shoulder pain, 29 trials met inclusion criteria, 20 involving sham controls (n = 5,230) and 18 non-sham controls (n = 14,597). For sham controls, we analysed non-needle sham, penetrating sham needles and non-penetrating sham needles. For non-sham controls, we analysed non-specified routine care and protocol-guided care. Using meta-regression we explored impact of choice of control on effect of acupuncture.Acupuncture was significantly superior to all categories of control group. For trials that used penetrating needles for sham control, acupuncture had smaller effect sizes than for trials with non-penetrating sham or sham control without needles. The difference in effect size was -0.45 (95% C.I. -0.78, -0.12; p = 0.007), or -0.19 (95% C.I. -0.39, 0.01; p = 0.058) after exclusion of outlying studies showing very large effects of acupuncture. In trials with non-sham controls, larger effect sizes associated with acupuncture vs. non-specified routine care than vs. protocol-guided care. Although the difference in effect size was large (0.26), it was not significant with a wide confidence interval (95% C.I. -0.05, 0.57, p = 0.1).Acupuncture is significantly superior to control irrespective of the subtype of control. While the choice of control should be driven by the study question, our findings can help inform study design in acupuncture, particularly with respect to sample size. Penetrating needles appear to have important physiologic activity. We recommend that this type of sham be avoided.

BACKGROUND:Recent evidence shows that acupuncture is effective for chronic pain. However we do not know whether there are characteristics of acupuncture or acupuncturists that are associated with better or worse outcomes. METHODS:An existing dataset, developed by the Acupuncture Trialists' Collaboration, included 29 trials of acupuncture for chronic pain with individual data involving 17,922 patients. The available data on characteristics of acupuncture included style of acupuncture, point prescription, location of needles, use of electrical stimulation and moxibustion, number, frequency and duration of sessions, number of needles used and acupuncturist experience. We used random-effects meta-regression to test the effect of each characteristic on the main effect estimate of pain. Where sufficient patient-level data were available, we conducted patient-level analyses. RESULTS:When comparing acupuncture to sham controls, there was little evidence that the effects of acupuncture on pain were modified by any of the acupuncture characteristics evaluated, including style of acupuncture, the number or placement of needles, the number, frequency or duration of sessions, patient-practitioner interactions and the experience of the acupuncturist. When comparing acupuncture to non-acupuncture controls, there was little evidence that these characteristics modified the effect of acupuncture, except better pain outcomes were observed when more needles were used (p=0.010) and, from patient level analysis involving a sub-set of five trials, when a higher number of acupuncture treatment sessions were provided (p

Many journals now require authors share their data with other investigators, either by depositing the data in a public repository or making it freely available upon request. These policies are explicit, but remain largely untested. We sought to determine how well authors comply with such policies by requesting data from authors who had published in one of two journals with clear data sharing policies.We requested data from ten investigators who had published in either PLoS Medicine or PLoS Clinical Trials. All responses were carefully documented. In the event that we were refused data, we reminded authors of the journal's data sharing guidelines. If we did not receive a response to our initial request, a second request was made. Following the ten requests for raw data, three investigators did not respond, four authors responded and refused to share their data, two email addresses were no longer valid, and one author requested further details. A reminder of PLoS's explicit requirement that authors share data did not change the reply from the four authors who initially refused. Only one author sent an original data set.We received only one of ten raw data sets requested. This suggests that journal policies requiring data sharing do not lead to authors making their data sets available to independent investigators.