Dealing with Messy Data: Analyzing Pre- and Post-test Assessment Results in the Real World. AIR 1999 Annual Forum Paper.

This paper presents a model for analyzing and drawing inferences from pre- and post-assessment data that are not as clean as might be desired. The model was applied to comparison of freshman and senior scores on the Academic Profile and Short Form at Cardinal Stritch University (Wisconsin). As of 1997-98, 891 pre-tests had been administered to new students and 514 post-tests to completing seniors. However, some seniors students were never post-tested, some students took the test more than once, and some students didn't take the test seriously. This study eliminated scores that were greater or less than two standard deviations from the gross means, then recalculated means and standard deviations. Then separate t-tests were compared: (1) pre- and post-test scores of students who had taken both tests (matched pairs); (2) scores of students who had taken only the pre-test or post-test (independent samples); and (3) comparisons of classic predictors of college success (college entrance exam scores, high school grades). Comparison of matched pairs indicated significant differences favoring the post-tests as did comparison of the independent samples. However, several predictor measures for pre-test takers were higher than for post-test takers, an unexpected finding for which possible interpretations are offered. (DB)