DESIGN AND SUBSTANCE IN EDUCATIONAL RESEARCH: ADOLESCENT ATTAINMENTS, A CASE IN POINT.

Much non-experimental educational research has relied upon cross-sectional designs, while many of the theories which motivate such inquiry imply some sort of developmental sequencing of the relationships of interest. This paper explores the practical implications of this disjuncture between perspective and practice by comparing the conclusions reached in the estimation of a recursive school process model of educational goal formation across several design options. Selecting out appropriate subsets of students from the repeated cross-sectional samples obtained in the Educational Testing Service's Study of Academic Prediction and Growth, we contrast: parallel cross-sectional and longitudinal analyses: longitudinal analyses which vary in the grade-level at which key independent variables are measured; and longitudinal analyses which do and do not take account of inter-temporal stability in the dependent variable(s). In general, we find that longitudinal results differ from cross-sectional: longitudinal differ from longitudinal depending upon the timing of variable measurement: and longitudinal differ from longitudinal depending on the adequacy of the controls employed. In typical research practice only one such set of results would he presented and, judging from the available literature, this most often would derive from the conceptually least adequate of the designs considered in our exercises. These comparisons urge caution in the uncritical acceptance of cross-sectional school process research. More generally, they underscore the complexity of design decisions in educational research and the importance of the investigators framing of the issues in informing those decisions. [ABSTRACT FROM AUTHOR]