Do case vignettes accurately reflect antibiotic prescription?

Background: Antibiotic prescription is frequently inappropriate in hospitals. Our objective was to evaluate whether the quality of antibiotic prescription could be measured using case vignettes to assess physicians' knowledge.
Methods: The study was conducted in 2 public teaching hospitals, where 33/41 units and 206/412 physicians regularly prescribing antibiotics to inpatients agreed to participate. A cross-sectional survey of knowledge was performed using 4 randomly assigned case vignette sets. Curative antibiotic prescriptions were then evaluated using standard criteria for appropriateness at initiation (day 0), after 2-3 days of treatment (days 2-3), and at treatment completion. We compared knowledge of the physicians with their observed prescriptions in the subset of 106 physicians who completed the case vignettes and prescribed antibiotics at least once.
Results: The median global case vignette score was 11.4/20 (interquartile range, 8.9-14.3). Of the 483 antibiotic prescriptions, 314 (65%) were deemed appropriate at day 0, 324 (72%) on days 2-3, and 227 (69%) at treatment completion. Prescriptions were appropriate at all 3 time points in only 43% of patients. Appropriate prescription was positively and independently associated with emergency prescription on day 0, documented infection on days 2-3, and 1 of the 2 hospitals at treatment completion. In addition, appropriateness at the 3 evaluation times was positively associated with prescription in anesthesiology or the intensive care unit. Case vignette scores above the median were significantly and independently associated with appropriate antibiotic prescription on days 2-3 and at treatment completion.
Conclusions: Case vignettes are effective for identifying physicians or hospitals whose knowledge and practice regarding antibiotic prescription require improvement.