The Limits of Reasoning: Students' Evaluations of Anecdotal, Descriptive, Correlational, and Causal Evidence

Students' evidence-based reasoning was examined across two studies. In Study 1, students were asked to evaluate newspaper excerpts including anecdotal, descriptive, correlational, and causal evidence provided in support of causal claims as well as to justify their quality ratings for two of these excerpts. In Study 2, students' justifications for quality ratings were further probed and students were asked to select the criterion that they considered to be most important to consider in evaluating information. Key findings included that while students were fairly effective at discounting anecdotal evidence relative to evidence that was descriptive, correlational, and causal, students did not seem to distinguish among these three latter evidence types, when these were provided in support of causal claims. While students cited a variety of justification criteria for their quality evaluations, they were found to rate methods-related factors as more important to consider when evaluating quantitative, rather than anecdotal, evidence.