Testing Validity Inferences of the Science Motivation Questionnaire II Scores Using a Rasch Analysis Framework

Motivation is a central variable in science education research. The Science Motivation Questionnaire II (SMQ II) is an instrument designed to measure student motivation to learn science. Nearly all peer-reviewed studies (n = 26) of the SMQ II have used Classical Test Theory frameworks despite known limitations with this approach. The present study seeks to evaluate the psychometric properties of the SMQ II using a Rasch Analysis framework. We explored three questions: (1) How well do the scales and items on the SMQ II function when analyzed in the context of the Rasch model? (2) Does time of the school year impact the psychometric properties of the instrument? And (3) How do the five motivational constructs differ by gender and grade level? We administered the SMQ II to a sample of students (n = 1,728) at two points in time. In order to evaluate instrument dimensionality, a Principle Component Analysis of Rasch residuals was performed. In addition, AIC/BIC values were generated and compared across different dimensional arrangements. Person reliability, item reliability, and item fit values were calculated for each dimension. Wright maps were also generated in order to compare person-item alignments. Finally, dataset stacking was used to align scores from the June and January samples and compare motivation patterns across gender and grade band for each of the five dimensions. Results of the dimensionality tests indicated that a five-dimensional model was better supported than a unidimensional model. Item and person reliabilities were acceptable for all dimensions. Infit and outfit MNSQ statistics for items revealed acceptable values in nearly all cases. Wright Maps for the five dimensions indicated a lack of items at the upper end of the logit scales. Rasch score differences across time, gender, and grade level were apparent, most notably for grade nine. For the Self-Determination and Career Motivation dimensions, grade nine participants had lower measures compared to the other grades. The Grade Motivation dimension had the highest mean scores, and the Career Motivation dimension had the lowest mean scores. These findings advance our understanding of the strengths and weaknesses of a core measure of student motivation to learn science. Overall, the SMQ II instrument functions adequately. However, several measurement problems were documented that must be addressed before using this instrument in comparable participant populations. Multiple analyses (e.g., separation values, Wright maps) revealed a lack of measurement precision for this participant population. Moreover, a few items are problematic must be modified or eliminated. Collectively, these findings suggest that the SMQ II will produce adequate but error-prone measures of student motivation. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]