Search

To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects.An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects.Meta-analyses from the Cochrane Pregnancy and Childbirth Database.The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding.Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects (P.001). Odds ratios were exaggerated by 41% for inadequately concealed trials and by 30% for unclearly concealed trials (adjusted for other aspects of quality). Trials in which participants had been excluded after randomization did not yield larger estimates of effects, but that lack of association may be due to incomplete reporting. Trials that were not double-blind also yielded larger estimates of effects (P = .01), with odds ratios being exaggerated by 17%.This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials.

In Reply. —We thank Drs ter Riet and Kessels for attempting to explain the similar sample sizes we found in treatment and control groups. Unfortunately, their suggestion of widespread use of a form of restricted randomization does not shed any new light on our findings. In our original article, we stated, "The strong tendency for the comparison groups to be of equal or similar sizes may be explained by unreported use of (1) restriction, usually blocking...." That explanation includes the model postulated by ter Riet and Kessels, which is just one of many different forms of restriction. Ter Riet and Kessels observe that our "model does not reflect randomized trial conduct in practice." Indeed, that just restates our main argument. The authors of the 96 reports stated or (more usually) implied use of simple, unrestricted randomization. Thereby, they, not we, specified the form of the model used. Furthermore, our approach

Recent empirical evidence supports the importance of adequate randomization in controlled trials. Trials with inadequate allocation concealment have been associated with larger treatment effects compared with trials in which authors reported adequate allocation concealment. While that provides empirical evidence of bias being interjected into trials, trial investigators rarely document the sensitive details of subverting the intended purpose of randomization. This article relates anonymous accounts of deciphering assignment sequences before allocation based on experiences acquired from epidemiologic workshops for physicians. These accounts run the gamut from simple to intricate operations, from transillumination of envelopes to searching for code in the office files of the principal investigator. They indicate that deciphering is something more frequent than a rare occurrence. These accounts prompt some methodological recommendations to help prevent deciphering. Randomized controlled trials appear to annoy human nature—if properly conducted, indeed they should. ( JAMA . 1995;274:1456-1458)

A RANDOMIZED controlled trial (RCT) is the most reliable method of assessing the efficacy of health care interventions.1,2Reports of RCTs should provide readers with adequate information about what went on in the design, execution, analysis, and interpretation of the trial. Such reports will help readers judge the validity of the trial. There have been several investigations evaluating how RCTs are reported. In an early study, Mahon and Daniel3reviewed 203 reports of drug trials published between 1956 and 1960 in theCanadian Medical Association Journal. Only 11 reports (5.4%) fulfilled their criteria of a valid report. In a review of 45 trials published during 1985 in three leading general medical journals, Pocock and colleagues4reported that a statement about sample size was only mentioned in five (11.1%) of the reports, that only six (13.3%) made use of confidence intervals, and that the statistical analyses tended to

Objective. —To assess the methodologic quality of approaches used to allocate participants to comparison groups in randomized controlled trials from one medical specialty. Design. —Survey of published, parallel group randomized controlled trials. Data Sources. —All 206 reports with allocation described as randomized from the 1990 and 1991 volumes of four journals of obstetrics and gynecology. Main Outcome Measures. —Direct and indirect measures of the adequacy of randomization and baseline comparisons. Results. —Only 32% of the reports described an adequate method for generating a sequence of random numbers, and only 23% contained information showing that steps had been taken to conceal assignment until the point of treatment allocation. A mere 9% described both sequence generation and allocation concealment. In reports of trials that had apparently used unrestricted randomization, the differences in sample sizes between treatment and control groups were much smaller than would be expected due to chance. In reports of trials in which hypothesis tests had been used to compare baseline characteristics, only 2% of reported test results were statistically significant, lower than the expected rate of 5%. Conclusions. —Proper randomization is required to generate unbiased comparison groups in controlled trials, yet the reports in these journals usually provided inadequate or unacceptable information on treatment allocation. Additional analyses suggest that nonrandom manipulation of comparison groups and selective reporting of baseline comparisons may have occurred. ( JAMA . 1994;272:125-128)

In Reply.— Dr Iffy's continued concerns about the effects of induced abortion, particularly DE he also selectively references those American studies that he believes support his contentions. In the interest of space, we refer the reader to a recent review article 4 that synthesized all the available data on the effects of induced abortion on reproductive function. It comes to a conclusion different from Dr Iffy's. His assessment of the influence of follow-up information on the rate of abortion complications is erroneous. Women who suffer complications after legally induced abortion are more (not less) likely than women without any complications to notify the facility performing

Abortions performed late in pregnancy are associated with higher death and complication rates, are more expensive, and are emotionally more taxing for patients and staff than abortions performed early in pregnancy. To examine potential correlates of late abortion, we studied 1,066 women having abortions at two Washington, DC, abortion facilities. We found the following variables to be statistically significantly related to the gestational stage of pregnancy at the time a woman seeks an abortion: history of irregular periods, level of education, type and number of pregnancy symptoms volunteered, moral feelings of the woman about abortion, confided suspicion of pregnancy to another person, knowledge of the legality of abortion, contraceptive use, pregnancy symptoms elicited by professional personnel, and length of time the sexual partner was known. Yet, these variables are responsible for only a small percentage of the problem, since much variation in gestational age was unexplained by our analysis. This result and the finding that most of the important determinants of delay were individually oriented factors strongly imply that the problem of late abortions is one that will not be greatly influenced by public health interventions. ( JAMA 244:44-48, 1980)

Although uterine perforation is a potentially life-threatening complication of curettage abortion, little is known about the risk factors associated with uterine perforation or how to prevent it. Using linear logistic regression, we analyzed 67,175 curettage abortions performed at 13 institutions in the United States from 1975 through 1978. The overall incidence of confirmed perforation was 0.9 per 1,000 abortions. Performance of the abortion by a resident rather than by an attending physician was a powerful risk factor for perforation (relative risk, 5.5; 95% confidence interval, 3.3 to 9.2). Use of Laminaria for dilation had a protective effect, although this effect was not statistically significant (relative risk, 0.17; 95% confidence interval, 0.02 to 1.2). Among factors beyond the control of the physician, advancement of gestational age and previous delivery were significant risk factors. Use of Laminaria and performance of the abortion by an attending physician considerably lowered the risk of uterine perforation.