Search

Construct maps and related item type frameworks, which respectively describe and identify common patterns in student reasoning, provide teachers with tools to support instruction built upon students' intuitive thinking. We present a methodological approach--embedded within common assessment development processes--for developing such construct maps and item type frameworks which entails (1) articulating a construct map, (2) developing diagnostic item types, (3) comparing item type difficulties, and (4) modifying or providing evidence in support of the construct map. We present an example focused on students' proportional reasoning based on its centrality to mathematical success. The results further inform our construct map for students' proportional reasoning and provide a diagnostic item-type framework.

Emotion recognition (ER) involves understanding what others are feeling by interpreting nonverbal behavior, including facial expressions. The purpose of this study is to evaluate the psychometric properties of a web-based social ER assessment designed for children in kindergarten through third grade. Data were collected from two separate samples of children. The first sample included 3,224 children and the second sample included 4,419 children. Data were calibrated using Rasch dichotomous model. Differential item and test functioning were also evaluated across gender and ethnicity. Across both samples, we found consistent item fit, unidimensional item structure, and adequate item targeting. Analyses of differential item functioning (DIF) found six out of 111 items displaying DIF across gender and no items demonstrating DIF across ethnicity. The analyses of person measure calibrations with and without DIF items yielded no evidence of differential test functioning (DTF) across gender and ethnicity groups in both samples. [For the corresponding grantee submission, see ED602750.]

The Child Observation Record (COR), initially developed in 1993 by HighScope Educational Research Foundation, is an observation-based instrument that provides systematic assessment of young children's knowledge and abilities in all major areas of development. Teachers or caregivers spend a few minutes each day writing brief notes or "anecdotes" that objectively describe significant episodes of young children's activities. The anecdotes are then classified and scored according to various COR categories, items, and levels, providing a comprehensive portrait of each child's developmental gains and the progress of the group as a whole. In 2012, the COR was revised to span the developmental range from infancy and toddlerhood through kindergarten. The validation evidence for the revised instrument, the COR Advantage, was gathered in two phases--an initial validation phase and the multi-state implementation validation phase. The results from both phases show that the COR Advantage generates scores which are internally consistent and inferences made from scores that are valid for (a) documenting the developmental trajectories of children from birth through kindergarten in all key areas of children's development, and (b) capturing developmental milestones of children from varied backgrounds with diverse abilities when used by early childhood professionals.

Proportional reasoning is important to students' future success in mathematics and science endeavors. More specifically, students' fluent and flexible use of scalar and functional relationships to solve problems is critical to their ability to reason proportionally. The purpose of this study is to investigate the influence of systematically manipulating the location of an integer multiplier--to press the scalar or functional relationship--on item difficulty and student solution strategies. We administered short-answer assessment forms to 473 students in grades 6-8 (approximate ages 11-14) and analyzed the data quantitatively with the Rasch model to examine item accessibility and qualitatively to examine student solution strategies. We found that manipulating the location of the integer multiplier encouraged students to make use of different aspects of proportional relationships without decreasing item accessibility. Implications for proportional reasoning curricular materials, instruction, and assessment are addressed.

Students who complete kindergarten with an inadequate knowledge of basic mathematics concepts and skills will continue to experience difficulties with mathematics throughout their elementary and secondary years and may be at increased risk for math disabilities. There is a critical need to identify students experiencing difficulties in mathematics in the early elementary grades and to provide immediate and targeted instruction to remediate these deficits. Most early math screening tools focus on only a single skill, resulting in an incomplete picture of student performance and limited predictive validity. To address this need, we are developing a multiple-gating system of math assessment, the Primary Math Assessment (PMA), that both screens and provides diagnostic information in six domains. In this study, we present the results of the development and validation of items across the domains that will comprise the PMA. Multidimensional Rasch models were used to estimate theoretically plausible dimensionality structures. Parsimony fit indices supported the six-dimensional model as the most generalizability model for the PMA data and supports reporting of six separate scores.

Measurement error is a common theme in classical measurement models used in testing and assessment. In classical measurement models, the definition of measurement error and the subsequent reliability coefficients differ on the basis of the test administration design. Internal consistency reliability specifies error due primarily to poor item sampling. Rater reliability indicates error due to inconsistency among raters. For estimates of test-retest reliability, error is attributed mainly to changes over time. In alternate-forms reliability, error is assumed to be due largely to variation between samples of items on test forms. Rasch models can also compute reliability estimates of scores under different test situations. The authors therefore present the Rasch perspective on calculating reliability (measurement error) and present Rasch measurement model programs to compute the various reliability estimates. (Contains 5 tables and 1 note.)

This study describes the use of generalizability theory (GT) and many-facet Rasch measurement (MFRM) to evaluate psychometric properties of responses obtained from an assessment designed to measure complex problem-solving skills. The assessment revolved around the school activity of kickball. The task required of each student was to decide on a team T-shirt based on visual characteristics of the T-shirts and written descriptions of other T-shirt characteristics. Forty-four fourth-grade students, comprising the control group of a longitudinal research project to foster complex problem-solving skills of disadvantaged urban youth, participated in this study. Results indicate both measurement techniques agree on the relative magnitudes of variation among the facets but differ on how to handle the sources of variation.

Extended work in comparing item fit statistics and principal component analysis as tools for assessing the unidimensionality requirements of Rasch models and demonstrated methods to explore how violations of the unidimensionality requirement influence person measurement through 2 studies involving simulated rating scale data and responses to a scale by 317 undergraduates. Results can help researchers determine the impact of multidimensionality. (SLD)

Discussed information available in the context of Rasch measurement that may be useful in optimizing rating scales and demonstrated the process in studies involving 330 fourth and fifth graders and 668 fourth and fifth graders. Results provide evidence for the generalizability of the findings and the usefulness of the scale with samples from the same population. (SLD)

Used Rasch measurement to study the psychometric properties of data obtained from a newly developed Diabetes Questionnaire designed to measure diabetes knowledge, attitudes, and self-care. Responses of 26 diabetes patients to the English version of the questionnaire and 24 patients to the Spanish version support the cross-form equivalence and allow quantitative comparison across versions. (SLD)

Examined a process for unifying two teacher self-efficacy scales, providing a common metric for communicating and comparing results. Data from 118 education students indicated that items from the two scales defined a unidimensional construct. Correlations between person measures from the two scales demonstrated their strengths for measuring portions of the self- efficacy variable. Raw score to interval measure conversion tables provide researchers a common metric for interpretation and analysis. (SM)

Outlines methods in Rasch measurement that are used to gather evidence for reliability and validity and attempts to articulate how these methods may be linked with current views of reliability and validity. (SLD)

Studied the dimensionality of responses to the Study Skills Self-Efficacy Scale (M. Ramirez and S. Owen, 1991) using exploratory factor analysis and investigated the use of Rasch measures in differentiating between groups of students who report being academically successful or at risk. Results for 550 college students indicate that responses define three related dimensions, and that Rasch measures were able to differentiate between the groups. (SLD)

Explored the possibility of developing a common metric for measuring goal orientation by simultaneously calibrating four commonly used goal orientation scales to see if item responses met the unidimensionality requirement. Results for 305 psychology students in France yield a common metric for interpretation and statistical analyses. (SLD)

Describes problems in score reporting with the True Score model, defines the Rasch measurement unit (the logit), reviews transformations of the logit metric, and provides examples of score reporting procedures. Uses dichotomous data from an examination taken by 126 Ph.D. students and polychotomous data from a self-efficacy assessment completed by 16 war veterans with Post Traumatic Stress Disorder to illustrate these issues. (SLD)

Explored the dimensionality of responses to the Adult Behavior Checklist-Revised, a screening assessment for Attention Deficit Hyperactivity Disorder (ADHA) in college students. Responses from 317 undergraduates were used, with 8 judges involved in standard setting. Findings show secondary variables that may have clinical implications. (SLD)

The factor structure of the "Diagnostic and Statistical Manual of Mental Disorders" (DSM-IV) criteria for attention deficit hyperactivity disorder (ADHD) is evaluated in a sample of 1,503 college students. The Adult Behavior Checklist is evaluated as a screening instrument. Results support the extension of ADHD criteria for diagnosis to college students. (Author/EMK)

This study assessed teachers' beliefs about the effectiveness of various instructional media across a diverse set of learning situations. Subjects were 87 preservice teachers at a major northeastern university. A confirmatory factor analysis supported two latent constructs representing beliefs about traditional and technology-based instructional media. Fit indices demonstrated excellent model fit to the data. (AEF)

Rasch methodology, often used in educational research, was used to assess psychometric limitations of a scale designed to measure fear of falling in older persons. Simultaneous calibration of scale items improved its lack of scale definition. Implications for assessment of self-efficacy perceptions and planning treatment programs are discussed. (SLD)